MATB24

Prof. Michael Cavers

W1 (3.1)

Def. Vector Space

A non-empty set V of objects called vectors, along with rules for
- adding any 2 vectors v and w in V
- $\R$
Must satisfy closure properties
- $\fa v, w \in V, v + w \in V$
- $\fa r \in \R, v \in V, rv \in V$
Must satisfy the following axioms for vector addition
1. Associative
  $(u + v) + w = u + (v + w)$
2. Commutative
  $u + v = v + u$
3. An additive identity exists
  $\exists$ $\in V$ 0 + v = v $\fa v \in V$
4. An additive inverse exists
  $\exists$ $-v \in V$ s.t. v + (-v) = 0
Must satisfy the following axioms for scalar multiplication
1. Distributes over vector addition
  $r\.(v+w) = (r\. v) + (r\. w)$
2. Distributes over scalar addition
  $(r+s)\. v = (r\. v) + (s\. v)$
3. Associative
  $r\.(s\. v) = (r\. s)\. v$
4. A multiplicative identity exists
  $1\. v = v$

Note: vectors could refer to matrices, polynomials, functions, power series, etc.

Q $V = \{\v v \in \R^n | \t{components of } \v v \t{ sum to 0}\}$ with usual operations. Is V a vector space?
A $\v u, \v v \in V$ $\v u = \[ u_1\\ \vdots \\ u_n\], \v v = \[ v_1\\ \vdots \\ v_n\]$
Check closure under addition:
$\v u + \v v = \sumto n(u_i + v_i) = \sumto n u_i + \sumto n v_i = 0 + 0 = 0 \in V$
Check closure under scalar mult:
$r\v u = \sumto n(r u_i) = r\sumto n u_i = r\.0 = 0 \in V$
$\implies$ $\implies$ V is a vector space.

Q $V = \R^2$ $a\\b\] + \[ c\\d\] = \[ a+c\\ b+d+1$
A $x\\ y\] + \[ a\\ b\] = \[ a\\ b\] \implies \[ x+a\\ y+b+1\] = \[ a\\ b\] \implies x = 0, y = 1 \implies \vec0 = \[ 0\\ -1$

To define a vector space, need to specify 4 parts:

Objects that make up the set V
The field where scalars come from (if the field of scalars is the set of real #s, then V is a real vector space)
How to add vectors
How to multiply scalars with objects in V

$\oplus, \odot$ are vector spaces.

Notation	Definition
$\R^n$	set of vectors of length n with real entries
$\Complex^n$	set of vectors of length n with complex entries
$P$	set of all polynomials in x with real coefficients
$P_n$	$P$ $\leq$ n
$M_{mn}$	set of all real matrices of size m(rows) by n(columns)
$F$	$\R$

Thm 3.1 (Elementary Properties of Vector Spaces)

Every vector space V has the following properties:

$\v 0$ is unique for all vectors in V
$-\v v$ is unique for each vector in V
$u+v = u+w \implies v = w$
$0\v v = \v0, \fa \v v \in V$
$r\v0 = \v0, \fa r \in \R$
$(-r)v = r(-v) = -(rv), \fa r\in \R,v \in V$

Proof

$\exists$ 2 $x$ $y$ $x + v = v$ $y + v = v$ .
$y\in V$ $v = y$ $x + v = v \implies x + y = y$
$x\in V$ $v = x$ $y + v = v \implies y + x = x$
$x + y = y + x \implies y = x \implies \v 0$ is unique
$v + (-v) = 0$ $v + u = 0$ .
$u = u+0 = u+[v+(-v)] = (u+v)+(-v) = 0+(-v) = (-v)$
$u = -v \implies$ additive inverse is unique
$u + v$ = $u + w$
$v = 0 + v = (-u + u) + v = -u + (u + v)$ = $-u + (u + w) = (-u + u) + w = 0 + w = w$
$\implies v = w$
$\v0 + 0\v v = 0\v v = (0 + 0)\v v = 0\v v + 0\v v$ .
$\implies \v0 = 0\v v$
$r\v v + r\v0 = r(\v v + \v 0) = r\v v = r\v v + \v 0$ .
$\implies r\v 0 = \v 0$
$r\v v + (-r\v v) = (r + (-r))\v v = 0\v v = \v0 \implies -(r\v v) = (-r)\v v$
$\v v + (-\v v) = \v 0 \implies r\v v + r(-\v v) = \v 0 \implies -(r\v v) = r(-\v v)$
$\implies -(r\v v) = (-r)\v v = r(-\v v)$

W2 (3.2)

Vector Space vs Real Space

Concepts for real space extend to vector spaces
Except that vector spaces could be spanned by an infinite set of vectors

E.g. $M = \{1, x, x^2, ..., x^n, ...\}$
$\implies$ $\iff$ P = span(M)
M generates P and so P is NOT finitely generated

Def. Linear Combination

$\v x$ $\v x_1, \v x_2, ... , \v x_k \in V$ $\exists$ $c_1, c_2, ... c_k \in \R$ $\v x = c_1\v x_1 + ... + c_k\v x_k$

Note linear combinations must be of a finite number of vectors (otherwise called Schauder basis)

Def. Span

$sp(X)$ is the set of all linear combinations of elements in X, where X is a subset of V.

$V = sp(X)$ , then the vectors in X span or generate V.

Q $1+x^2 \in sp({1+x, x+x^2, 2+x^2})$ ? Assume standard polynomial addition and scalar mult.
A $1+x^2$ $a(1+x) + b(x+x^2) + c(2+x^2)$
$1+x^2 = a + ax + bx + bx^2 + 2c + cx^2$
$\implies\bc 1=a+2c\\ 0=a+b\\ 1=b+c\ec$ (derived using the coefficients)
$\implies\bc a=-1/3\\ b=1/3\\ c=2/3\ec\implies$ $1+x^2$ is in the span of the set.

Thm (Span of a subset is a subspace)

$X$ $V$ $sp(X)$ $V$ .

Proof

Since X is non-empty, sp(X) must be non-empty
$r_1, r_2 \in \R$ $v_1, v_2 \in sp(X)$
$v_1, v_2 \in sp(X), \ex a_1,...,a_n\in\R, b_1,...b_n\in \R, x_1, ..., x_n \in X$ s.t.
$v_1 = a_1x_1 + … + a_nx_n$
$v_2 = b_1x_1 + … + b_nx_n$
$r_1v_1 + r_2v_2 = r_1(a_1x_1 + … + a_nx_n) + r_2(b_1x_1 + ... + b_nx_n)$
$= (r_1a_1 + r_2b_1)x_1 + ... + (r_1a_n + r_2b_n)x_n \in sp(X)$

Non-empty, closed under additional and scalar mult -> span of X is a subspace of V.

E.g. $X = \set{1, x, x^2} \in P$
$(1+x)+(1+x^2)=x^2 + x + 2$ , which is still a linear combination of vectors in X, so we have closure under addition.
$r(1+x) = r + rx$ , which is a multiple of - again - linear combinations of vectors in X, so we have closure under mult.

Corollary (Not covered in class)

$sp(X)$ $V$ $X$ .

Proof

$v_1, ..., v_n$

$M = span(v_1, …, v_n)$

$v_i = 0\. v_1 + ... + 1\. v_i + ... + 0\.v_n$ $1 \le i \le n$ ,

$v_i \in span(v_1, ..., v_n)$ $M \subseteq span(v_1, ... v_n)$

$v_i \in M$ and every subspace is a vector space with closure properties,

$a_1v_1 + ... + a_nv_n \in M$ $span(v_1, … v_n) \subseteq M$

$M = span(v_1, …, v_n)$

Def. Subspace

$V$ $U$ $V$ $U$ $V$ $U$ $V$ .

E.g. Counter examples
$\R^2$ that are closed under addition but not scalar multiplication:
$S = \{\[x\\y\]:x, y \ge 0\}$ $\not\geq 0\ so\notin S$
$S=\{(n,0)\in\mathbb{R}^2:n\in\mathbb{Z}\}$ $\notin \Z$
$S=\{(r,s)\in\mathbb{R}^2:r,s\in\mathbb{Q}\}$ $\pi$ $\notin \Q$

$\R^2$ which is closed under scalar multiplication but not addition:
$\{\[x\\y\]:x = 0\ or\ y = 0\}$

$U$ is a subspace

$U$ is a vector space using the definition (too much work)
Use the span theorem (probably easiest)
Use the subspace test

Thm 3.2 (The Subspace Test)

$U$ $V$ $V$ if

$\v0 \in U$ (Sometimes omitted as it is implied by 3)
$U$ is closed under vector addition
$U$ is closed under scalar multiplication

$a\v x + b\v y \in U$

Q $M_n$ (the vector space of all nxn matrices)?
A No. The sum of 2 invertible matrices may not be invertible (violates 2).
Also, the 0 matrix is not invertible (violates 1).

Def. Linear (In)dependence

Let X be a set of vectors in a vector space V. (X can be an infinite set.)

$r_1\v v_1 + ... + r_k\v v_k = \v0$ $r_i \neq 0$ $\v v_i \in X$ for i = 1, … , k.

If such a dependence relation exists, then X is linearly dependent. Otherwise, X is linearly independent.

If X is finite, then the following definition applies:

$\{\v x_1, ..., \v x_k\}$ $\v x_1, ..., \v x_k \in V$ $c_1\v x_1 + ... + c_k\v x_k = \v 0$ $c_1 = ... = c_k = 0$

Note: { } is independent (vacuous statement)
There is no collection of vectors from { } satisfying a dependence relation because there is nothing in the set at all
$\v x: \underbrace{\v x\t{ is the linear combination of vectors in \{ \}}}_{\t{an empty sum}}$ }
$\v 0$ ).

Q $\{\v x, \v y, \v z\}$ $\{x + y, 2x + z, y - 5z\}$ is also linearly independent.
A $c_1(x+y)+c_2(2x+z)+c_3(y-5z) = \v 0$
$c_1 = c_2 = c_3 = 0$
$c_1x + c_1y + 2c_2x + c_2z + c_3y - 5c_3z = \v 0$
$(c_1 + c_2)x + (c_1 + c_3)y + (c_2 - 5c_3)z = \v 0$
$\{\v x, \v y, \v z\}$ $\bc c_1 + 2c_2 = 0\\ c_1 + c_3 = 0\\ c_2 - 5c_3 = 0 \ec \implies c_1, c_2, c_3 = 0 \implies independent$

Q $\R \to \R$ (with usual operations).
$X = \{\sin^2x, \cos^2x, 1\}$ linearly independent?
A $(1)sin^2x + (1)cos^2x + (-1)(1) = 0$

Q $X = \{\sin x, \cos x\}$ linearly independent?
A X is linearly independent.
$a\sin x + b\cos x = 0$ $\forall x$
$x = 0, \pi/2$ since it holds for all x.)
$a\sin0 + b\cos0 = 0 \implies b = 0$
$a\sin(\pi/2) + b\cos(\pi/2) = 0 \implies a = 0$
$\because a, b = 0$ $\therefore$ X is linearly independent
If we find a non-trivial solution (a or b or both != 0), NO conclusion can be made.
To generate more equations, we can take derivatives first then plug in points.

To check if a set is linearly independent:

If it cannot be observed, set up the dependence relation equation as the first step.

If the set spans a space with a dimension lower than the cardinality of the set (i.e. dim(span of the set) < |set|), then the set cannot be linearly independent.

If given a set of polynomials, put the coefficients as column vectors in a matrix, reduce to REF, and solve the system of equations.

Other ways:

create a system of equations by plugging in dif values of x (# values = # unknowns) and solve
differentiate the dependence relation, plug in the same value of x each iteration
expand and rearrange in terms of a linear independent set, then the coefficients must be 0, solve
see if the set contains the zero vector

Def. Basis

A subset X of V is a basis for V if:

X spans V
X is linearly independent

Def. Dimension

Let V be a finitely generated vector space. The number of elements in a basis for V is the dimension of V, dim(V).

Note: When giving definitions, "a" and "the" imply different things - "the" implies that the object described is unique.

Q Is dimension well defined? I.e. Can 2 bases have different sizes?

A It is well defined since any 2 bases of V must have the same number of elements. This is true by the following theorem.

Thm (Dimension and Linear Independence)

$V$ is a vector space of dimension n. Then any subset X with size greater than n is linearly dependent.

Proof

$B = \set{v_1, ..., v_n}$

$X = \set{x_1, ..., x_m}$ $m>n$

$B$ $x_j$ $B$ $x_i = \d\sum_{j=1}^n a_{ji}v_j, i\in\set{1,…, m}$ .

$x_1 = a_{11}v_1 + ... + a_{n1}v_n$

$\vdots$

$x_m = a_{1m}v_1 + ... + a_{nm}v_n$

$\exists$ $(\lambda_1, ..., \lambda_m)$ $a_{11} &... &a_{1m}\\ \vdots &\ddots &\vdots\\ a_{n1} &... &a_{nm}\]\[\lambda_1\\ \vdots\\ \lambda_m\] = \[0\\ \vdots\\ 0$

Since m > n, we have more unknowns than equations, so there must be a non-trivial (non-zero) solution, i.e. the lambda vector must be non-zero.

$\sumto m \lambda_i x_i = 0$

$\sumto m \lambda_i x_i = \sumto m \lambda_i (\sum_{j=1}^n a_{ji}v_j) = \sum_{j=1}^n(\sumto m \lambda_i a_{ji})v_j \stackrel{optional}{=} \sum_{j=1}^n (\lambda_1a_{j1} + ... + \lambda_m a_{jm})v_j = \sum_{j=1}^n \lambda_1a_{j1}v_j + ... + \lambda_m a_{jm}v_j$

$a_{ji}$ $\lambda_i$ cannot all be 0, so there exists at least one non-zero coefficient in the dependence relation
so X is not linearly independent.

$\qed$

E.g. $\R^n$ $\left\{ \[ 1\\ 0\\ 0\\ \vdots\\ 0 \], \[ 0\\ 1\\ 0\\\vdots\\ 0 \], ..., \[ 0\\ 0\\ 0\\ \vdots\\ 0\] \right\}$ $\dim(\R^n) = n$
E.g. $M_{m, n}$ $\dim(M_{m,n}) = m\. n$
E.g. $M_{n}$ $\dim(M_n) = n$
E.g. $P_n$ $\{1, x, ..., x^n\}$ $\dim(P_n) = n + 1$

Note $\infty$ (not defined in the textbook).
Note $\{\v0\}$ $\null$ which has dimension 0.
$\v 0$ $c_1 \. \v 0 = \v 0$ $c_1 = 1$
$\{\v 0\}$ since a basis cannot be a linearly dependent set

$\subseteq$ $\subseteq$ Spanning Set)

$U$ $U$ any $U$ . (Be careful which ones you add - to preserve independence.)
$U$ $U$ . (Be careful which ones you delete to preserve span.)

Thm 3.3 (Unique Representation)

$B$ $V$ .

$B$ $V$ $\v v$ $V$ uniquely $\v v = r_1\v b_1+...+r_k\v b_k$ $r_i \in \R$ $\v b_i \in B$ .

Proof for Thm 3.3

When B is finite:

$(\Rightarrow)$ $\ex \v v$ that can be expressed the following 2 ways:

$\ba \v v &= r_1\v b_1 + ... + r_k\v b_k \\ \v v &= s_1\v b_1 + ... + s_k\v b_k\ea \implies \v 0 = (r_1 - s_1)\v b_1 + ... + (r_k - s_k)\v b_k$

$r_1 - s_1 = 0, ... , r_k - s_k = 0 \implies r_1 = s_1, ... ,r_k = s_k \implies$ $\v v$ has unique representation.

$\Leftarrow$ ) Suppose every vector can be uniquely expressed in terms of B. This implies V = span(B).

$\v 0$ $r_1\v b_1 + ... + r_k \v b_k = \v 0 \implies r_1 = ... = r_k = 0 \implies$ B is lin ind.

B spans V and is linearly independent, so B must be a basis.

When B is infinite:

pad two different expressions of a vector (in terms of a finite number of elements in B) with zeros
so that the two expressions use the same finite subset of vectors from B
As a result, B now represents this finite subset (of the infinite set) used in our representations of the vector v.

Def. Finitely Generated

$V = sp(X)$ where X is finite.

Q Which of these vector spaces is finitely generated?
$\R \to \R$ ?
The vector space of all polynomials.
The vector space of all polynomials of degree less than or equal to n.
The vector space of all 2×2 matrices.
A $\set{1, x^2, ..., x^n}$
$E_{ij}$ where the ij-th entry = 1 and all other = 0

Thm (Shortcut for proving basis)

Let B be a set of m vectors from V. If |B| = dim(V), then showing one of (B is lin ind, B spans V) will suffice.

Proof $\dim(V) = m$ , then any lin ind B is a basis of V.)

B is a basis for V if it spans V and is lin ind. Lin ind is given, so just need to show it spans V.

$\ex v \in V, v\notin sp(B) \implies \dim(B \u \set v) = m+1$ .

This contradicts the fact that any set with >m elements cannot be lin ind in a vector space of dimension m.

Proof $\dim(V) = m$ , then any B spanning V is a basis of V.)

B is a basis for V if it spans V and is lin ind. We know B spans V, so just need to show lin ind.

Suppose that B is not lin ind. Then at least 1 of its elements can be written as a lin combo of the other elements.

$B'$ , which would have the same span as B. B' would have m-1 elements.

Continue deleting until lin ind. Since the set has the same span as B, it must be a basis for V.

$\dim(V) = m$ .

Q $U = \{p(x) \in P_2: p(1)=0\}$ $U$ $P_2$ $U$ .
A $p(x) \in U$ $p(1) = 0$
$p(x) = ax^2 + bx + c$ $a(1)^2 + b(1) + c = 0 \implies a + b + c = 0 \implies \bc a = -s-t\\ b = s\\ c = t\ec$ $s, t\in \R$
$p(x) = (-s-t)x^2 + sx + t$
$U = \{(-s-t)x^2 + sx + t: s, t\in \R\} = \{s(-x^2 + x) + t(-x^2 + 1):s, t\in\R\} = sp(\{-x^2+x, -x^2+1\})$
$U$ $P_2$ .
$\{-x^2+x, -x^2+1\}$ $U$ $\dim(U) = 2$ .

W3 (3.3, 3.4 pg. 213-216)

Recall: $T:\R^n\to\R^m$ $A_{mn} = \left[T(\v e_1)|...|T(\v e_n)\right]$ $\{\v e_1, ..., \v e_n\}$ $\R^n$ $\v x \in \R^n$ $T(\v x) = A\v x$ .

Ordered Bases

We can give an order to every basis. Note: To emphasize the basis is ordered, we use ( ) instead of { }.

E.g. $\R^3$ $\v e_1, \v e_2, \v e_3$ $\left\{\[1\\0\\0\], \[0\\1\\0\], \[0\\0\\1\]\right\}$
$(e_1, e_2, e_3), (e_1, e_3, e_2), (e_2, e_1, e_3), (e_2, e_3, e_1), (e_3, e_1, e_2), (e_3, e_2, e_2)$

Def. Coordinate Vector

$B = (\v b_1, ..., \v b_n)$ be an ordered basis for a vector space V.

$\v v = c_1\v b_1 + ... + c_n\v b_n$ $\v v$ relative to the basis B $\v v_B = \[c_1\\ \vdots\\c_n\]$

By the unique representation theorem, this vector is well defined.

E.g. $\v v = 2x^3 - 3x + 1 \in P_2$
$B = (1, x, x^2)$ $\v v_B = \[1\\-3\\2\]$
$B = (x, 2x^2 , 1)$ $\v v_B = \[-3\\2\\1\]$

Thm (Coordinatization of Vector Spaces)

$V$ $n$ $\R^n$ .

$\v b_1, ...,\v b_n$ ).

$\v v \in V$ unique $V$ $\R^n$

$\R^n$ $V$

$\R^n$

Proof

$\v v, \v w \in V, t \in \R$ $\R^n$ ops:

$(\v v + \v w)_B = \v v_B + \v w_B$

Suppose (by unique representation theorem) we have vectors v and w:

$\v v = r_1\v b_1 + ... + r_n\v b_n \implies \v v_B = (r_1,...,r_n)$

$\v w = s_1\v b_1 + ... + s_n \v b_n \implies \v w_B = (s_1, ..., s_n)$

$(\v v + \v w)_B = (r_1\v b_1 + ... + r_n\v b_n + s_1\v b_1 + ... + s_n\v b_n)_B$

$= ((r_1 + s_1)\v b_1 + ... + (r_n + s_n)\v b_n)_B$

$= \[r_1 + s_1\\ \vdots\\ r_n + s_n\] = \[r_1\\ \vdots\\ r_n\] + \[s_1\\ \vdots\\ s_n\] = \v v_B + \v w_B$

$(t\v v)_B = t\v v_B$

$(t\v v)_B = (t(r_1\v b_1 + ... + r_n\v b_n))_B = (tr_1\v b_1 + ... + tr_n\v b_n)_B = [tr_1, ..., tr_n] = t[r_1, ..., r_n] = t\v v_B$

Def. Isomorphic

$V$ $\R^n$ $V$ .

$V$ $W$ $V$ $W$ $V\cong W$ .

E.g. $P_n \cong \R^{n+1}$ $M_{m,n} \cong P_{mn-1}\cong\R^{mn}$

Thm (Dimension Determines Structure)

$V$ $\R$ $\dim(V) = n$ $V$ $\R^n$

$\oplus, \odot$ $V$ $W$ $\iff$ dim(V) = dim(W)

This is contrary to other mathematical structures like groups, rings, etc.

Def. Linear Transformation

$T:V\to W$ is a linear transformation if

T preserves addition
$\v v_1, \v v_2 \in V$ $\underbrace{T(\v v_1 + \v v_2)}_{in\ V} = \underbrace{T(\v v_1) + T(\v v_2)}_{in\ W}$
T preserves scalar multiplication
$\v V \in V, k \in \R$ $T(k\.\v v) = k\.T(\v v)$

$T(a_1\v v_1 + ... + a_k \v v_k) = a_1T(\v v_1) + ... + a_kT(\v v_k)$

Fact $\v b_i$

Proof $T(\v b_i) = T'(\v b_i) \fa \v b_i \in B$

$\v v \in V$ $\ex \v b_1, ..., \v b_k \in B$ $r_1, .., r_k \in \R$ $\v v = r_1\v b_1 + ... + r_k\v b_k$

$\ba T(\v v) &= T(r_1\v b_1 + ... + r_k\v b_k)\\ &= r_1T(\v b_1) + ... + r_kT(\v b_k)\\ &= r_1T'(\v b_1) + ... + r_kT'(\v b_k)\\ &= T'(r_1b_1 + ... + r_kb_k)\\ &= T'(\v v)\ea$

$T, T'$ are the same transformation

Other facts:

T preserves zero vector
$T(\v 0_V) = \v 0_W$
Proof $T(\v 0) \stackrel{r=0}= T(0\v0) = 0T(\v 0) = \v 0$
T preserves additive inverse (and thus substraction)
$T(-\v v) = -T(\v v)$

E.g. $T: M_{n, n} \to \R$ $T(A) = tr(A)$ is a linear transformation
Recall $tr(A) = \d\sum_{i=1}^n a_{ii}$
T preserves addition
$T(A+B) = tr(A+B) = tr(A) + tr(B) = T(A) + T(B)$
T preserves scalar mult
$T(kA) = tr(kA) = k\ tr(A) = k\ T(A)$

E.g. $T: M_{n, n}\to\R, n>1$ $T(A) = \det A$ is not a linear transformation
T preserves addition
Consider A = B = I
$T(A) + T(B) = T(I) + T(I) = \det I + \det I = 1 + 1 = 2$
$T(A+B) = T(I+I) = \det(I+I) = \det(2I) = 2^n$
Recall Determinant of a diagonal matrix is the product of the diagonal entries
The above are not equal if n > 1
T preserves scalar mult - no need to check

E.g. Derivatives and integrals are linear transformations
$T(f) = f'$
$T(f+g) = (f+g)' = f' + g' = T(f) + T(g)$
$T(rf) = (rf)' = r(f') = rT(f)$
$T(f) = \d\int_a^b f(x)dx$
$T(f+g) = \d\int_a^b[f(x) + g(x)]dx = \int_a^bf(x)dx + \int_a^bg(x)dx = T(f) + T(g)$
$T(rf) = \d\int_a^b rf(x)dx = r\int_a^bf(x)dx = rT(f)$

Def. Domain, Codomain

$V$ $W$ $T:V\to W$ be a linear transformation.

$V$ domain $T$ $W$ codomain $T$ .

Def. Image, Inverse Image

image $\Im(T) = \{T(\v v): \v v\in V\} = T[V]$

$\Im(T)$ inverse image $\set{\v v:T(\v v)\in W'} = T\inv[W']$

Def. Range, Kernel / Null Space

$T[V]$ is the range of T (also image of V under T).

kernel $\v 0_W$ $\ker(T) = \set{\v v: T(\v v) = \v 0_W} = T\inv[\set{\v 0_W}]$

$\ker(T)$ homogeneous transformation equation $T(\v v) = \v 0_W$

Thm (Rank-Nullity)

$\dim(V) = rank(T) + \text{nullity}(T) = \dim(\Im(T)) + \dim(\ker(T))$

$\v 0_W$

Proof

$\{\v v_1, ..., \v v_k\}$ $\ker(T)$ $\dim(\ker(T)) = k$

$V:\{\v v_1, ..., \v v_k, \v w_1, ..., \v w_m\}$ $\dim(V) = k + m$

$\dim(V) = k+m, \dim(\ker(T)) = k$ $\dim(\Im(T)) = m$

Do so by finding a basis for the image:

$\v v \in V$ $\v v = a_1 \v v_1 + ... + a_k \v v_k + b_1\v w_1 + ... + \v b_m\v w_m$ ,

$\Im(T) = \{T(\v v):\v v \in V\} = sp(\set{\underbrace{T(\v v_1), ..., T(\v v_k)}_{T(\v v_i) = \v 0_W}, T(\v w_1), ..., T(\v w_m)}) = sp(\{T(\v w_1), ..., T(\v w_m)\})$

$\{T(\v w_1), ..., T(\v w_m)\}$ $\lambda_1\.T(\v w_1) + ... + \lambda_m\.T(\v w_m) = \v 0_W$

$\implies T(\lambda_1\v w_1 + ... + \lambda_m \v w_m) = \v 0_W$ since T is linear

$\implies \lambda_1 \v w_1 + ... + \lambda_m \v w_m \in \ker(T)$

$\implies \lambda_1 \v w_1 + ... + \lambda_m \v w_m = \mu_1\v v_1 + ... + \mu_k \v v_k$ $\set{\v v_1, ..., \v v_k}$ $\ker(T)$

$\implies \lambda_1 \v w_1 + ... + \lambda_m \v w_m + (-\mu_1)\v v_1 +...+ (-\mu_k) \v v_k = \v 0_V$

$\implies \lambda_1 = ... = \lambda_m = \mu_1 = ... = \mu_k = 0$ $\{\v v_1, ..., \v v_k, \v w_1, ..., \v w_m\}$ is lin ind (it is a basis)

$\{T(\v w_1), ..., T(\v w_m)\}$ is lin ind, which means it is a basis for the image.

$\dim(\image(T)) = m, \dim(V) - \dim(\ker(T)) = k$

Technique to Find Bases for Image(T)

$\ker(T)$
extend it to a basis for V
$\image(T)$

E.g. $A = \[0&1\\1&0\]$ $T:M_{2,2}\to M_{2,2}$ $T(X) = XA - AX \fa X\in M_{2,2}$
$\image(T)$ using the above technique.
$\ker(T)$
$X = \[a&b\\c&d\]\in\ker(T)\iff \[a&b\\c&d\]\[0&1\\1&0\] - \[0&1\\1&0\]\[a&b\\c&d\] = \[0&0\\0&0\] \iff \[b-c&a-d\\d-a&c-b\] = \[0&0\\0&0\]$
$\implies \bc b-c=0\\a-d=0\\d-a=0\\c-b=0 \ec \implies \left[\begin{array}{rrrr|r}0&1&-1&0&0\\1&0&0&-1&0\\-1&0&0&1&0\\0&-1&1&0&0\end{array}\right] \implies \left[\begin{array}{rrrr|r}1&0&0&-1&0\\0&1&-1&0&0\\0&0&0&0&0\\0&0&0&0&0\end{array}\right]\implies \bc c=s\\d=t\\a=t\\b=s\ec$
$\implies \ker(T) = \{\[t&s\\s&t\]:s, t\in\R\} = \{s\[0&1\\1&0\]+t\[1&0\\0&1\]:s, t\in \R\} = sp$\[0&1\\1&0\],\[1&0\\0&1\]$$
Extend it to a basis for V
$M_{2,2}$
$\{\[0&1\\1&0\],\[1&0\\0&1\], \[1&0\\0&0\],\[0&1\\0&0\]\}$ (This is not the only way)
$\image(T)$
$\{T$\[1&0\\0&0\]$, T$\[0&1\\0&0\]$\} = \{\[0&1\\0&-1\],\[1&0\\0&-1\]\}$

E.g. $T: P_{4} \rightarrow P_{4}$ $T(p)=x p^{\prime}(x)-p(x)$ $T$ $T$ .

Method 1: finding a matrix rep rel. to ordered bases and observing the null/column space
$B=B^{\prime}=\left(1, x, x^{2}, x^{3}, x^{4}\right)$ $P_{4}$ .
$T(1) = x(0) - 1 = -1$
$T(x) = x(1) - x = 0$
$T(x^2) = x(2x) - x^2 = x^2$
$T(x^3) = x(3x^2) - x^3 = 2x^3$
$T(x^4) = x(4x^3) - x^4 = 3x^4$
$A$ $T$ $B, B^{\prime}$ is
$A=\left[\begin{array}{ccccc} \mid & \mid & \mid & \mid & \mid \\ T(1)_{B^{\prime}} & T(x)_{B^{\prime}} & T\left(x^{2}\right)_{B^{\prime}} & T\left(x^{3}\right)_{B^{\prime}} & T\left(x^{4}\right)_{B^{\prime}} \\ \mid & \mid & \mid & \mid & \mid \end{array}\right]=\left[\begin{array}{ccccc} -1 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 2 & 0 \\ 0 & 0 & 0 & 0 & 3 \end{array}\right]$
null space $A$ $\{[0,1,0,0,0]\}$ $\operatorname{ker}(T)$ $\{x\}$ .
Note: dim(ker(T)) = dim(null space(A))
column space $A$ $\{[-1,0,0,0,0],[0,0,1,0,0],[0,0,0,2,0],[0,0,0,0,3]\}$ $\operatorname{im}(T)$ $\left\{-1, x^{2}, 2 x^{3}, 3 x^{4}\right\}$
Method 2: finding a basis for the kernel of T and extending it to get a basis for the image of T
$\ker(T) = T\inv[\set{\v 0_W}]$ $T(p)=0$
$p(x) = ax^4 + bx^3 + cx^2 + dx + e$ $T(p)=x p^{\prime}(x)-p(x) = 0 \implies x\left(4 a x^{3}+3 b x^{2}+2 c x+d\right) - \left(a x^{4}+b x^{3}+c x^{2}+d x+e\right)=0$
$3ax^4 + 2bx^3 + cx^2 - e = 0 \implies a = b = c = e = 0,\ d = k, k \in \R$
$[4 a=a, 3 b=b, 2 c=c, d=d, 0=e]$ )
$p = kx \implies \ker(T)=\{k x: k \in \mathbb{R}\}=\t{sp}(x) \implies \{x\}$ $\operatorname{ker}(T)$ .
$P_{4}$ $1, x^{2}, x^{3}, x^{4}$ to the set.
$\operatorname{im}(T)$ $\left\{T(1), T\left(x^{2}\right), T\left(x^{3}\right), T\left(x^{4}\right)\right\}=\left\{-1, x^{2}, 2 x^{3}, 3 x^{4}\right\}$

Def. Row Space

Motivation $\R^n$ $\{\v v_1, \v v_2, ..., \v v_m\}$ .
What operations can be performed on this basis while preserving its span and linear independence?

Swap 2 vectors
Multiply a vector by a non-zero scalar
Add a scalar multiple of 1 vector to another

$\{\v v_1, \v v_2, ..., \v v_m\}$ are the rows of a matrix, these are simply elementary row operations!

$A$ row space $\R^n$ spanned by its rows, denoted row(A)

Facts

Elementary row operations do not change the row space
In general, the rows of A may be linearly dependent

Thm The non-zero rows of any REF is a basis for row(A)

E.g. $U = sp\{\[1\\0\], \[1\\-1\], \[2\\-2\]\}$ using row space

$A = \[1&0\\1&-1\\2&-2\] \implies row(A) = sp\{\[1\\0\]^T, \[1\\-1\]^T, \[2\\-2\]^T\}$

$A = \[1&0\\1&-1\\2&-2\] \to \[1&0\\1&-1\\0&0\] \to \[1&0\\0&-1\\0&0\] \to \[1&0\\0&1\\0&0\]$

$\{[1, 0], [0, 1]\} \implies U$ $\{\[1\\0\], \[0\\1\]\}$

E.g. $A = \[1&-1&1&3&2\\2&-1&1&5&1\\3&-1&1&7&0\\0&1&-1&-1&-3\]$

$A$ $\[1&-1&1&3&2\\0&1&-1&-1&-3\\0&0&0&0&0\\0&0&0&0&0\] \implies \{[1, -1, 1, 3, 2], [0, 1, -1, -1, -3]\}$ is a basis

So dim(row(A)) = 2

Def. Column Space

column space $\R^m$ spanned by its columns, denoted col(A);
$A$ $A^T$ $col(A)$ $A^T$ to REF

Note $\image(A) = col(A)$

E.g. $A = \[1&2&-1&-2&0\\2&4&-1&1&0\\3&6&-1&4&1\\0&0&1&5&0\]$

$A$ $1&2&0&3&0\\0&0&1&5&0\\0&0&0&0&1\\0&0&0&0&0$

Since (col 2) = 2(col 1), and (col 4) = 3(col 1) + 5(col 3),

$\{\[1\\2\\3\\0\], \[-1\\-1\\-1\\1\], \[0\\0\\1\\0\]\}$

Note We can't use columns of the REF for a basis, since row operations mess up the vectors!

Strategy for finding basis of row(A) and col(A)

reduce A to REF = R
A basis for row(A) is formed by rows of R containing leading 1's
A basis for col(A) is formed by cols of A corresponding to cols of R with leading 1's

Def. Rank

$A$ is the number of leading 1'sREF $rank(A) \le \min(m, n)$
$rank(A) = \dim(row(A)) = \dim(col(A)) = \dim(\image(A))$

$rank(A) = 2 = \dim(row(A)) = \dim(col(A))$

$m = 2, n = 3: \[1&0&0\\0&1&0\]$

$rank(A) = m \iff col(A) = \R^m \iff$ $A$ $\R^m$ $A$ $\R^n$

$m = 3, n = 2: \[1&0\\0&1\\0&0\]$

$rank(A) = n \iff row(A) = \R^n \iff$ $A$ $\R^n$ $A$ $\R^m$

E.g. Can a 5 x 6 matrix have linearly independent rows/cols?

$1&0&0&0&0&0\\0&1&0&0&0&0\\0&0&1&0&0&0\\0&0&0&1&0&0\\0&0&0&0&1&0$

$\R^5$ $\R^5$ . The 6th column can be written as a lin combo of the first 5. The 6th variable can be anything.

Rows can be linearly dependent or independent.

Note The converse: if # rows > # columns, the rows must be lin dep, the cols may be lin dep.

E.g. $A$ $rank(A) = m$ $m \le n$

$rank(A) \le n$ $rank(A) = m$ $m \le n$

E.g. $A$ $\dim(null(A)) \ge 4$

$rank(A) \le 5$ $rank(A) = n - \dim(null(A)) \implies 9 - \dim(null(A)) \le 5 \implies 4 \le \dim(null(A))$

Note $A_{m,n}$ $rank(A) + \t{nullity}(A) = n$

W4 (remainder of 3.4)

Thm 3.7 (Lin transf preserves subspaces)

$T:V\to V'$ be a linear transformation.

Proof $S$ $V$ $T[S]$ $V'$ )

Subspace test:

$T[S]$ contains the zero vector
$S$ $\v 0_V \in S \implies T(\v 0_V) \in T[S] \stackrel{\text{linearity of T}}\implies \v 0_{V'} \in T[S]$
$T(\v s_1) + T(\v s_2) \in T[S]$ $T(\v s_1), T(\v s_2) \in T[S]$ $\v s_1, \v s_2 \in S$
$S$ $\v s_1 + \v s_2 \in S \implies T(\v s_1) + T(\v s_2) \in T[S] \stackrel{\text{linearity of T}}\implies T(\v s_1 + \v s_2) \in T[S]$
$rT(\v s) \in T[S]$ $T(\v s) \in T[S]$ $\v s \in S$
$S$ $r\v s \in S \implies T(r\v s) \in T[S] \stackrel{\text{linearity of T}}\implies rT(\v s) \in T[S]$

Proof $S'$ $V'$ $T\inv[S']$ $V$ )

$T\inv[S']$ contains the zero vector
$S'$ $\v 0_{V'} \in S' \implies T\inv(\v 0_{V'}) \in T\inv[S'] \stackrel{\text{linearity of }T\inv}\implies \v 0_V \in T\inv[S']$
$\v v_1 + \v v_2 \in T\inv[S']$ $\v v_1, \v v_2 \in T\inv[S']$ $T(\v v_1), T(\v v_2) \in S'$
$S'$ $T(\v v_1) + T(\v v_2) \in S' \implies T\inv(T(\v v_1) + T(\v v_2)) \in T\inv[S'] \stackrel{\text{linearity of T}}\implies \implies \v v_1 + \v v_2 \in T\inv[S']$
$r\v v \in T\inv[S']$ $\v v \in T\inv[S']$ $T(\v v) \in S'$
$S'$ $rT(\v v) \in S' \implies T\inv(rT(\v v)) \in T\inv[S'] \stackrel{\text{linearity of T}}\implies r\v v \in T\inv[S']$

Def. Onto & 1-1

$T:V\to W$ is a linear transformation

Onto (surjective)	1-1 (injective)
$\v w \in W, \ex\v v \in V$ $T(\v v) = \v w$	$\v v_1, \v v_2 \in V$ $T(\v v_1) = T(\v v_2) \implies \v v_1 = \v v_2$ $\v v_1 \neq \v v_2 \implies T(\v v_1) \neq T(\v v_2)$
all elements in W have been mapped to at least once	all elements in W have been mapped to at most once
$\fa \v w \in W, T(\v v) = \v w$ has at least 1 solution	$\fa \v w \in W, T(\v v) = \v w$ has at most 1 solution
$\fa \v w \in W, A\v v = \v w$ is consistent	$\fa \v w \in W, A\v v = \v w$ has a unique sol'n or is inconsistent
A has a pivot in every row	A has a pivot in every column
The columns of A span W	The columns of A are lin ind $A\v x = \v 0$ $\ker(T) = \set{\v 0_V}$
T preserves spanning sets $V = sp\{\v v_1, ..., \v v_k\}$ $W = sp\{T(\v v_1), ..., T(\v v_k)\}$	T preserves lin ind $\{\v v_1, ..., \v v_k\}$ $\{T(\v v_1), ..., T(\v v_k)\}$ is a lin ind subset of W.
dim(range of T) = dim(W)	dim(range of T) = dim(V)

Note If T is both onto and 1-1, then it is bijective (and it must be an isomorphism, so it's invertible).

$T:V\to W, \dim(V) = \dim(W), T(\v v) = A\v v = \v w$ has a unique solution (i.e. A has a pivot in every row and column)

Thm (Extra TB notes)

$T(\v p) = \v b$ for some particular vector in V.

$T(\v x) = \v b$ $\set{\v p + \v h|\v h\in\ker(T)}$ (h is the solution to the homogeneous transf eq)

This means:

$\ker(T) = \set{\v 0_V}$ $\v h = \v 0 \implies \v p + \v h = \v p \implies T(\v x) = \v b$ $\v p$ , so T is 1-1
$T(\v x) = \v 0_W$ $\ker(T) = \set{\v 0_V}$

$\iff \ker(T) = \v 0_V$ follows from this.

E.g. $y'' - 4y = x^2$ $\set{p+h|h\in\ker(T)}$
$T(y) = y'' - 4y$ $\v b = x^2$ )
$\v p$ $T(\v p) = \v b$ $y'' - 4y = x^2$
$y = \f {-x^2}4$ $-\f12 - 4y = -4y \implies -\f12 = 0$ which can't be true
$-\f12$ $y = \f{-x^2-\f12}4$ $-\f12 - 4(\f{-x^2-\f12}4) = x^2$ which holds
$p(x) = y = \f{-x^2-\f12}4$
$T(\v x) = \v 0$ $y'' - 4y = 0$
$h(x) = c_1e^{2x} + c_2e^{-2x} \iff \set{e^{2x}, e^{-2x}}$ $\ker(T) \iff h \in \ker(T)$
$y'' - 4y = x^2$ $h(x) + p(x) = c_1e^{2x} + c_2e^{-2x} + (\f{-x^2-\f12}4)$

Thm (Compositions of lin transf is a lin transf)

$T:U\to V$ $S:V\to W$ be linear transformations (U, V, W are vector spaces).

$S\circ T: U\to W$ is also a linear transformation

Proof $\v x, \v y \in V, a, b \in \R$

$S\circ T(ax + by) = a S\circ T(x) + b S\circ T(y)$

$S\circ T(ax + by) = S(T(ax+by)) = S(aT(x) + bT(y)) = aS(T(x)) + bS(T(y)) = a S\circ T(x) + b S\circ T(y)$

Def. Invertible Transformation

$T:V\to W$ $T\inv : W\to V$ $T\inv \circ T = I_V, T\circ T\inv = I_W$ $I_W$ $I_W(\v x) = \v x$

$T\inv$ is the inverse for T (using "the" because it is unique).

Fact $\iff$ T is an isomorphism

Thm (T is invertible if A is invertible)

$B=\left\{\mathbf{b}_{1}, \ldots, \mathbf{b}_{n}\right\}$ $V$ $C=\left\{\mathbf{c}_{1}, \ldots, \mathbf{c}_{n}\right\}$ $W$ $T: V \rightarrow W$ $T$ $[T]_{B}^{C}$ $\left.R_{B, C}\right)$ $T$ $B$ $C$ is an invertible matrix.

$\ex T\inv: W\to V$ $T\inv \circ T = I_V, T\circ T\inv = I_W$ .
$\left[T^{-1}\right]_{C}^{B}$ $T^{-1}$ $C$ $B .$
$[T\inv]_{C}^{B}[T]_{B}^{C}=I_{n}$ $[T]_{B}^{C}[T\inv]_{C}^{B}=I_{n}$ $[T]_{B}^{C}$ is an invertible matrix.
$[T]_{B}^{C}$ $A$ $A[T]_{B}^{C}=I_{n}$ $\[T\]_{B} ^{C} A=I_{n}$
$A$ $S: W \rightarrow V$ $[S]_{C}^{B}=A$ .
$S \circ T = I_V$ $T \circ S = I_W$ $W$ ), so S is an inverse for T, and T is invertible.

Thm 3.8 (T is invertible iff 1-1 and onto)

Proof $T:V\to V'$

$\Rightarrow$ ): invertibility implies 1-1 & onto

$\ex \v v \in V$ $T\inv(\v v') = \v v$ $v' \in V'$ $V'$ .

$v' = (T\circ T\inv)(\v v') = T(T\inv(\v v')) = T(\v v)$ $\v v_1 \neq \v v_2 \implies T(\v v_1) \neq T(\v v_2)$ . Hence T must be 1-1.

$\Leftarrow$ ): 1-1 & onto implies invertibility

$V'$ $\ex \v v \in V$ $T\inv(\v v') = \v v$ $v' \in V'$ $\v v$ must be unique.

$T\inv: V' \to V$ $T\inv(\v v') = \v v$ $\v v$ $T(\v v) = \v v'$

$(T\circ T\inv)(\v v') = T(T\inv(\v v')) = T(\v v) = \v v \implies T\circ T\inv = I_{V'}$

$(T\inv \circ T)(\v v) = T\inv(T(\v v')) = T\inv(\v v') = \v v \implies T\inv \circ T = I_V$

Hence T is invertible.

$T\inv$ is indeed a linear transformation:

Preserves addition
$T\inv(v_1' + v_2') = T\inv(T(v_1) + T(v_2)) = T\inv(T(v_1 + v_2)) = v_1 + v_2 = T\inv(v_1) + T\inv(v_2)$
Preserves scalar mult
$T\inv(rv_1') = T\inv(rT(v_1)) = T\inv(T(rv_1)) = rv_1 = rT\inv(v_1')$

Def. Isomorphism

$T:V\to W$ is an isomorphism if it is 1-1 and onto.

E.g. $M_{2, 2} \cong P_3$
$T:M_{2, 2} \to P_3$ that is 1-1 and onto:
$T$\[a&b\\c&d\]$ = a + bx + cx^2 + dx^3$
Show linearity (T preserves addition and scalar mult)
$\ba &T$r\[a&b\\c&d\]+s\[e&f\\g&h\]$\\ &= T$\[ra+se&rb+sf\\rc+sg&rd+sh\]$\\ &= (ra+se) + (rb+sf)x + (rc+sg)x^2 + (rd+sh)x^3\\ &= r(a + bx + cx^2 + dx^3) + s(e + fx + gx^2 + hx^3)\\ &= rT$\[a&b\\c&d\]$+sT$\[e&f\\g&h\]$ \ea$
$\ker(T) = \v 0$ )
$\ba T(A) = 0 &\iff T$\[a&b\\c&d\]$ = 0 + 0x + 0x^2 + 0x^3\\ &\iff a + bx + cx^2 + dx^3 = 0 + 0x + 0x^2 + 0x^3\\ &\iff a = b = c = d = 0\\ &\iff \ker(T) = \{\[0&0\\0&0\]\} \ea$
Show onto
$p(x) = q + rx + sx^2 + tx^3 \in P_3$ $M_{2, 2}$ that maps to it, namely
$A = \[q&r\\s&t\]$

Thm (Iso proof shortcut if same dim)

$T:V\to W$ a linear transformation.

$\dim V = \dim W$ , then T is an isomorphism iff T is OR(onto, 1-1).

$\dim(\image(T)) + \dim(\ker(T)) = \dim(V)$

$\implies$ invertible
$\dim(W) = \dim(V) = n$ $T$ $\ker(T) = \{\v 0\} \implies \dim(\ker(T)) = 0$
$\dim(\Im(T)) + 0 = n$ .
$\image(T)$ $W$ $n$ $\image(T) = W$ . Hence, T is onto.
Since T is both 1-1 and onto, it must be invertible.
$\implies$ invertible
$\dim(W) = \dim(V) = n$ $T$ $\image(T) = W \implies \dim(\image(T)) = n$
$n + \dim(\ker(T)) = n \implies \dim(\ker(T)) = 0 \implies \ker(T) = \{\v 0\} \implies T$ is 1-1.
Since T is onto and 1-1, it must be invertible.

Thm 3.9 (Coordinatization is an isomorphism)

$B = (\v b_1, ..., \v b_n)$

$T:V\to\R^n$ $T(\v v) = \v v_B$ is an isomorphism.

Proof

T is a lin transf since it preserves addition and scalar mult. as proved in W3: Coordinatization of Vector Spaces

T is an isomorphism:

$\v v_B$ $\v v$
$\R^n$

Note $\R^n$ is not unique, since there is a corresponding isomorphism for each choice of an ordered basis of V.

W5 (7.1, 7.2)

Def. Standard Matrix Rep of Lin Transf

$T:\R^n \to \R^m$

$T:\R^n \to \R^m$ $T(\v x) = A\v x$ .
$A_{m\times n} = [T(\v e_1)|...|T(\v e_n)] = T$I_{m\times n}$$ is the $T$
$\v e_j$ determines the structure of A
$A\v x = \v b$ $T(\v x) = \v b$

Q $T([x_1, x_2, x_3]) = [x_1 - x_2 + 3x_3,\quad x_1 + x_2 + x_3,\quad x_1]$
A $T:\R^n \to \R^m$ $A_{m\times n} = [T(\v e_1)|...|T(\v e_n)] = T$I_{m\times n}$$
$T:\R^3\to \R^3$ $A = \[1&-1&3\\1&1&1\\1&0&0\]$
$\v e_j$ ,
just write out the coefficients of the transformed input as row vectors

$T:V\to W$ $V, W$ are non-Euclidean vector spaces?

E.g. $\tau: M_{2, 2}\to P_3$ $\tau$\[a&b\\c&d\]$ = a + (a+c)x + (b-c)x^2 + dx^3$
$\tau$ $M_{2, 2}$ $P_3$
$M_{2, 2}: B = $\[1&0\\0&0\], \[0&1\\0&0\], \[0&0\\1&0\], \[0&0\\0&1\]$$
$P_3: C = (1, x, x^2, x^3)$
$\R^n$ , we use the following coord. transformations:
$R:M_{2, 2} \to \R^4$ $R$\[a&b\\c&d\]$ = \[a\\b\\c\\d\]$
$S:P_3 \to \R^4$ $S(a+bx+cx^2+dx^3) = \[a\\b\\c\\d\]$
$T:\R^4 \to \R^4$ $T$\[a\\b\\c\\d\]$ = \[a\\a+c\\b-c\\d\]$
$T = S\circ \tau\circ R\inv$
$\R^4 \to \R^4 \equiv \R^4 \to M_{2,2} \to P_3 \to \R^4$ , illustrated below)
$T:$
$A = T$\[1&0&0&0\\0&1&0&0\\0&0&1&0\\0&0&0&1\]$ = \[ 1&0&0&0\\ 1&0&1&0\\ 0&1&-1&0\\ 0&0&0&1\]$ $\tau$ with respect to B and C.
$\implies$ $\implies$ T is an isomorphism.
$\tau = S\inv \circ T \circ R \implies \tau$ is also an isomorphism.

Matrix Rep Notation

$B$ $B'$ $R_{B, B'}$ $R_{B, B'}(T)$ $[T]_B^{B'}$

The following denote the matrix rep of T inverse

$R_{C, B}(T\inv) = [R_{B, C}(T)]\inv$
$[T\inv]_C^B = ([T]_B^C)\inv$

The following denote S composed with T illustrated above

$S\circ T$ $\.$ (matrix for T)
$R_{B,D}(S\circ T) = R_{C, D}(S)\.R_{B, C}(T) \expl {outside must match}$
$[S\circ T]_B^D = [S]_C^D\.[T]_B^C \expl {bottom left and top right must match}$

change of basis $C_{B, B'}$ illustrated below

$R_{B, B'}(I)$ $I$ is the identity transformation
$[I]_B^{B'}$

Def. Matrix of T Rel. to Basis B

$B = (\v b_1, …, \v b_n)$

$T:V\to V$ $R_B = [T(\v b_1)_B \mid ... \mid T(\v b_n)_B]$ $B$

$T(\v v)_B = R_B \v v_B$

Def. Matrix of T Rel. to Bases B and B'

$B = (\v b_1, …, \v b_n)$ $B' = (\v b_1', ..., \v b_m')$ .

$T:V\to V'$ $\bar T:\R^n\to\R^m$ $\bar T(\v v_B) = T(\v v)_{B'}$ .

$\bar T$ $\v v_B$ $\v v$ $\v v$ rel. to B'

$B$ $B'$ $A = \left[\ T(\v b_1)_{B'}\ \ ...\ T(\v b_n)_{B'}\ \right]$

$B$ $B'$

$T(\v v)_{B'} = A(\v v_B) \fa \v v \in V$

I.e. the coord. of any transformed vector can be obtained by:
multiplying A (coord. of transformed basis vectors rel to B') with its coord. relative to B

Note $A\inv$ $T\inv$ $B', B$ .

Prove: $T(\v b_j)_{B'}$

$\v v = a_1 \v b_1 + ... + a_n\v b_n$ $\v v_B = \[a_1\\ \vdots\\ a_n\]$

$\v b_j = 0\v b_1 + ... + 1\v b_j + ... + 0\v b_n$ $[\v b_j]_B = \[0\\ \vdots\\ 1\\ \vdots\\ 0\] = \v e_j$ $[\v b_j]_B = \v e_j$

$T(\v v)_{B'} = A(\v v_B) \implies T(\v b_j)_{B'} = A[\v b_j]_B = A\v e_j = \[a_{11} &... &a_{1n}\\ \vdots &\ddots &\vdots\\ a_{m1} &... &a_{mn}\]\[0\\ \vdots\\ 1\\ \vdots\\ 0\] =$ j-th column of A

E.g. $R_{B, B'}$ $B = B' = (\sin x\cos x, \sin^2x, \cos^2x)$ $f'(x)$ , given
$f(x) = 3\sin x \cos x - 5\sin^2x + 7\cos^2 x$
$\ba R_{B, B'} &= \[T(\sin x \cos x)_{B'} &T(\sin^2 x)_{B'} &T(\cos^2 x)_{B'}\]\\ &= \[(-\sin^2x+\cos^2x)_{B'} &(2\sin x\cos x)_{B'} &(-2\sin x\cos x)_{B'}\]\\ &= \[0&2&-2\\-1&0&0\\1&0&0\] \ea$
$T(\v v)_{B'} = R_{B, B'}\v v_B$
$[f'(x)]_{B'} = R_{B, B'}[f(x)]_B = \[0&2&-2\\-1&0&0\\1&0&0\]\[3\\-5\\7\] = \[-24\\-3\\3\]$
$f'(x) = -24\sin x\cos x - 3\sin^2 x + 3\cos^2 x$

E.g. $T:P_3 \to M_{2,2}$ $T(ax^3 + bx^2 + cx + d) = \[2a-2b&-2b-3c\\ 2a-2b+d&-3c+d\]$
$B = (x^3, x^2, x, 1)$ $P_3$ ,
$B' = $\[1&0\\0&0\], \[0&1\\0&0\], \[0&0\\1&0\], \[0&0\\0&1\]$$ $M_{2,2}$
$R_{B, B'}(T), R_{B', B}(T\inv), T\inv$\[p&q\\r&s\]$$
$\ba R_{B, B'}(T) &= \[T(x^3)_{B'} &T(x^2)_{B'} &T(x)_{B'} & T(1)_{B'}\]\\ &= \[ \[2 &0\\ 2&0\]_{B'}& \[-2 &-2\\ -2&0\]_{B'}& \[0 &-3\\ 0&-3\]_{B'}& \[0 &0\\ 1&1\]_{B'} \]\\ &= \[ 2&-2&0&0\\ 0&-2&-3&0\\ 2&-2&0&1\\ 0&0&-3&1 \] \ea$
$R_{B', B}(T\inv) = \[ 2&-2&0&0\\ 0&-2&-3&0\\ 2&-2&0&1\\ 0&0&-3&1 \]\inv = \[ 1&-1/2&-1/2&1/2\\ 1/2&-1/2&-1/2&1/2\\ -1/3&0&1/3&-1/3\\ -1&0&1&0 \]$
$T\inv(v)_B = R_{B', B}(T\inv)(v_{B'})$
$\implies T\inv$\[p&q\\r&s\]$_B = R_{B', B}(T\inv)\[p\\q\\r\\s\] = \[p-\frac12q-\frac12r-\frac12s \\\frac12p-\frac12q-\frac12r+\frac12s \\-\frac13p + \frac13r - \frac13s \\-p+r\]^T$
$\implies T\inv$\[p&q\\r&s\]$ = (p-\f12q-\f12r-\f12s)x^3 + (\f12p-\f12q-\f12r+\f12s)x^2 + (-\f13p + \f13r - \f13s)x + (-p+r)$

Def. Change of basis matrix

If we have a vector space V with basis Bfind a new basis B' $C_{B, B'} = \[[\v b_1]_{B'} &... & [\v b_n]_{B'}\]$ change of basis formula $\v v_{B'} = C_{B, B'} \. \v v_B$

$\v v$ $\cdot$ $\v v$ in old basis)

Note $T(\v v)_{B'} = A\.\v v_B$ from thm 3.10. Note its similarity to the change of basis formula.

Note $C_{B', B} = C\inv_{B, B'}$

E.g. $B = (x^2 - 1, x^2 + 1, x^2 + 2x + 1), B' = (x^2, x, 1)$
$C_{B, B'} = \[(x^2-1)_{B'} &(x^2+1)_{B'} &(x^2+2x+1)_{B'}\] = \[1&1&1\\0&0&2\\-1&1&1\]$

$V = \R^n$ [new basis vectors | old basis vectors] $C_{B, B'}$ :

$\left[\begin{array}{ccc|ccc}\v b_1' &... &\v b_n' &\v b_1 &... &\v b_n\end{array}\right] \sim \left[\begin{array}{c|c}I & C_{B, B'}\end{array}\right]$

E.g. $V = \R^3$ $C_{B, B'}$ $B = (\v e_1, \v e_2, \v e_3)$ $B' = ([1, 1, 0], [2, 0, 1], [1, -1, 0])$ .
$\v v_{B'}$ $\v v = \[2\\3\\4\]$
$\baug{ccc|ccc} 1&2&1&1&0&0\\ 1&0&-1&0&1&0\\ 0&1&0&0&0&1 \eaug\sim \baug{ccc|ccc} 1&0&0&1/2&1/2&-1\\ 0&1&0&0&0&1\\ 0&0&1&1/2&-1/2&-1 \eaug$
$C_{B, B}$ $[\v e_i]_{B'}$ $1&0&0\] = \f12\[1&1&0\] + 0\[2&0&1\]+\f12\[1&-1&0$
$\v v_{B'} = C_{B, B'}\v v_B = \[1/2&1/2&-1\\0&0&1\\1/2&-1/2&-1\]\[2\\3\\4\] = \[-3/2\\ 4\\ -4/3\]$

Motivation for next topic

$T:P_2\to P_2$ $T(P) = p'$ is the derivative

$B = (1, x, x^2)$ $R_B = \[0&1&0\\0&0&2\\0&0&0\]$

$B' = (x+1, x-1, 2x^2)$ $\bc T(x+1) = 1 = 1/2 (x+1) - 1/2 (x-1) + 0(2x^2)\\ T(x-1) = 1 = above\\ T(2x^2) = 4x = 2(x+1) + 2(x-1) + 0(2x^2)\ec$

$R_{B'} = \[1/2&1/2&2\\-1/2&-1/2&2\\0&0&0\]$

$R_B, R_{B'}$ $T$ . Their similarities are :

both not onto with rank 2
both have det = 0, trace = 0
both have the eigen values {0, 0, 0}

These similarities are not a coincidence. Similar matrices have similar properties.

Def. Similar Matrices

$B = C\inv AC$

A and B would have the same rank, determinant, eigenvalues, trace, and change of basis matrix.

Note A and B are similar iff they are the matrix reps of the same linear transformation relative to different bases. (thm 7.1)

Thm 7.1 (Basis Change in A Vector Space)

$V$ $B, B'$ $T:V\to V$ $R_B$ $R_{B'}$ $R_{B'} = C\inv R_B C$ $C = C_{B', B}$

$C_{B, B'} R_B = R_{B'}C_{B, B'}$ which is illustrated below (right + down = down + right)

$C_{B, B'} = (C_{B', B})\inv$ $R_{B'} = C_{B, B'}R_B(C_{B, B'})\inv = (C_{B', B})\inv R_BC_{B', B}$

E.g. $T:P_2 \to P_2, T(p(x)) = p(x-1)$
$B = (x^2, x, 1), B' = (x, x+1, x^2-1)$
$R_B, R_{B'}, C$ $R_{B'} = C\inv R_B C$
$C_{B', B} = \[[\v b_1']_B &[\v b_2']_B &[\v b_3']_B\] = \[0&0&1\\1&1&0\\0&1&-1\]$
$\ba R_B &= \[T(x^2)_B &T(x)_B &T(1)_B\]\\ &=\[(x-1)^2_B &(x-1)_B &1_B\]\expl {shift x by 1}\\ &=\[(x^2-2x+1)_B &(x-1)_B &1_B\]\\ &= \[1&0&0\\-2&1&0\\1&-1&1\] \ea$
$\ba R_{B'} &=\[T(x)_{B'} &T(x+1)_{B'} &T(x^2-1)_{B'}\]\\ &=\[(x-1)_{B'} &(x)_{B'} &((x-1)^2-1)_{B'}\]\expl {shift x by 1}\\ &=\[(x-1)_{B'} &(x)_{B'} &(x^2-2x)_{B'}\]\\ &=\[2&1&-3\\-1&0&1\\0&0&1\] \expl {} \bc x-1 & =2(x) - 1(x+1) + 0(x^2-1)\\ x & =1(x) + 0(x+1) + 0(x^2-1)\\ x^2 - 2x & =-3(x) + 1(x+1) + (x^2+1)\ec\\ &= C\inv R_B C\\ \ea$

Shortcut to find Matrix Rep (Add basis E)

$S = I$ $W$ $B'$ $E$ $U = W$ and we have the following

$\implies R_{B,E} = C_{B', E}R_{B, B'}$

Rearrange the above to get:

$R_{B, B'}(T) = (C_{B', E})\inv R_{B,E}(T) = C_{E, B'}R_{B,E}(T)$
$\[T\]_B^{B'} = (\[I\]_{B'}^E)\inv\.\[T\]_B^E = \[I\]_E^{B'}\.\[T\]_B^E$

E.g. $V$ $P_3$ $T:P_3 \to P_3$ $T(p(x)) = x^2p''(x)$
$R_B$ $B = (1, 1+x, 1-x^2, 1+x^3)$

$E$ $(1, x, x^2, x^3)$ $R_{B, B} = (C_{B, E})\inv R_{B, E}$

$C_{B, E} = \[1&1&1&1\\0&1&0&0\\0&0&-1&0\\0&0&0&1\]$

Apply T to basis and decompose/express as lin combo:

$\begin{alignat}{3} &T(1) & &= 0 &&= 0(1) + 0(1+x) + 0(1-x^2) + 0(1+x^3)\\ &T(1+x) & &= 0 &&= 0(1) + 0(1+x) + 0(1-x^2) + 0(1+x^3)\\ &T(1-x^2) && = -2x^2 &&= -2(1) + 0(1+x) + 2(1-x^2) + 0(1+x^3)\\ &T(1+x^3) && = 6x^3 &&= -6(1) + 0(1+x) + 0(1-x^2) + 6(1+x^3) \end{alignat}$

$R_{B, E} = \[0&0&0&0\\0&0&0&0\\0&0&-2&0\\0&0&0&6\]$

Recall $[A|I]\to[I|A\inv] \expl{the row ops essentially multiply both sides by $A\inv$}$

$[A|C]\to[I|A\inv C]$

$R_{B, B} = (C_{B, E})\inv R_{B, E}$ $C_{B, E}|R_{B, E}\] \to \[I|(C_{B,E})\inv\.R_{B,E}\] = \[I|R_{B,B}$

$\baug{rrrr|rrrr} 1&1&1&1&0&0&0&0\\ 0&1&0&0&0&0&0&0\\ 0&0&-1&0&0&0&-2&0\\ 0&0&0&1&0&0&0&6 \eaug \to \baug{rrrr|rrrr} 1&0&0&0&0&0&-2&-6\\ 0&1&0&0&0&0&0&0\\ 0&0&1&0&0&0&2&0\\ 0&0&0&1&0&0&0&6 \eaug$

$R_B$ $T(\v v)$ $R_B \. \v v_B = T(\v v)_B$ )

Thm (Basis Change between 2 Vector Spaces)

$T:V\to W$ $V$ $A, A'$ $W$ $B, B'$ $A, B$ $\to A', B'$ using

$R_{A', B'} = C_{B, B'}R_{A, B}C_{A', A}$
$[T]_{A'}^{B'} = [I_W]_B^{B'}[T]_A^B[I_V]_{A'}^A$

Def. Eigen Values/Vectors

Given a transformation, an eigen vector is a non-zero vector that only gets stretched/scaled by a factor (not morphed)
$\ba A\v v &= \lambda \v v \quad\gr\text{where } \lambda \text{ is the scaling factor, called eigen value}\\ A\v v &= (\lambda I)\v v \quad\gr\text{Since LHS is matrix mult, turn RHS into matrix mult as well}\\ \quad (A-\lambda I)\v v &= \v 0 \quad\gr A - \lambda I\text{ transforms } \v v \text{ to }\v 0\\ \quad \det(A - \lambda I) &= 0 \quad\gr\text{Since the determinant is the scaling factor for a volume under a transformation, it must = 0 here} \ea$
To obtain the eigen value, solve the equation that arises from det = 0 (last line), called the characteristic polynomial
To obtain the eigen vectors, plug the eigen values back into the second to last line, and solve the system

$T:V\to V$ $V$ be a vector space

$\lambda$ eigen value $T$ $\exists$ $\v v \in V$ $T(\v v) = \lambda \v v$
$\v v$ eigen vector $\lambda$
- $\v v$ is unchanged/invariant under T
$\lambda$ $\v v$ $\lambda^k$ $A^k$ $\v v$

Fact $T:\R^n \to \R^n$ , then eigen values of T are eigen values of the standard matrix rep of T

E.g. $P_2$ $E = (1, x, x^2)$ .
$T:P_2\to P_2$ $\bc T(1) = 3 + 2x + x^2\\ T(x) = 2\\ T(x^2) = 2x^2\ec$
Find matrix for T relative to E:
$R_E = \[3&2&0\\2&0&0\\1&0&2\]$
Find eigen values by performing cofactor expansion down the 3rd column:
$\ba \det (R_E - \lambda I) &=\bd 3-\lambda &2 & 0\\ 2 &-\lambda & 0\\ 1 & 0 & 2-\lambda\ed\\ &=(2-\lambda) \bd 3-\lambda &2\\2 &-\lambda \ed\\ &= (2-\lambda)((3-\lambda)(-\lambda)-4)\\ &= (2-\lambda)(-3\lambda + \lambda^2 - 4)\\ &= (2-\lambda)(\lambda+1)(\lambda-4) \ea$
Eigenvalues $P_2$ Eigenvectors for T
$\lambda_1 = -1$ [-3, 6, 1] $p_1(x) = -3+6x+x^2$
$\lambda_2 = 2$ [0, 0, 1] $p_2(x) = x^2$
$\lambda_3 = 4$ [2, 1, 1] $p_3(x) = 2 + x + x^2$
Note $T$ $B = (-3 + 6x + x^2, x^2, 2 + x + x^2)$
$B$ $R_B = \lambda I = \[-1&0&0\\0&2&0\\0&0&4\]$

Eigenvalues	$P_2$	Eigenvectors for T
$\lambda_1 = -1$	[-3, 6, 1]	$p_1(x) = -3+6x+x^2$
$\lambda_2 = 2$	[0, 0, 1]	$p_2(x) = x^2$
$\lambda_3 = 4$	[2, 1, 1]	$p_3(x) = 2 + x + x^2$

Def. Eigen Space

$\lambda$ eigen space $E_\lambda$ $\lambda$ plus the zero vector (which isn't categorized as an eigenvector, even though it has the properties of one).

Note $E_\lambda = \{\v x : (A - \lambda I)\v x = \v 0\} = null(A - \lambda I) = \ker(A - \lambda I)$ $\R^n$ $\C^n$ ).

Def. Alg & Geo Multiplicities

Algebraic multiplicity $\lambda$ = its multiplicity (as a root) in the characteristic polynomial

Geometric multiplicity $\lambda$ $\dim(E_\lambda)$

Def. Diagonalizable

Def. $A_n$ $D = P\inv A P \iff A = PDP\inv$
$P$ $A_n$
Note $\lambda_i$ , and the column vectors of P are the eigen vectors
Thm $A_n$ is diagonalizable
$\iff$ $A_n$ $\R^n$ )
$\iff$ sum of (alg mult of eigen values of A) = n, and each eigen value has geo mult = alg mult
$\iff$ $\ex r$ $\R^n = E_{\lambda_1} \oplus...\oplus E_{\lambda_r}$
$\oplus$ $\R \oplus \R = \R^2$ )

E.g. $T:P_2 \to P_2$ $T(p) = p'$ $T(ax^2 + bx + c) = 2ax + b$ . Is T diagonalizable?

$B = (1, x, x^2)$ $P_2$

$[T]_B^B = R_{B, B} = \[0 &1 &0\\0 &0 &2\\ 0 &0 &0\]$

Does there exist 3 lin ind. eigen vectors for the matrix rep of T?

$[T]_B^B$ $p(\lambda) = \det([T]_B - \lambda I) = \bd-\lambda &1 &0 \\ 0 &-\lambda &\lambda\\ 0 &0 &-\lambda\ed = (-\lambda)^3 = -\lambda^3$

$\lambda = 0$ $[T]_B^B$ $\lambda_1 = \lambda_2 = \lambda_3 = 0$ )

$[T]_B^B - 0I = \v 0 \implies \[0&1&0\\0&0&2\\ 0&0&0\]\sim \[0&1&0\\0&0&1\\ 0&0&0\] \implies$ $r\[1\\0\\0\], r\ne0$

$E_\lambda = \{r\[1\\0\\0\]: r\in\R\} = sp(\[1\\0\\0\])$ and the geometric multiplicity of eigenvalue 0 is 1

$\dim(E_\lambda) \ne \dim$ (vector space), so T is not diagonalizable.

Note $[T]_{B'}$ which is the (diagonal) matrix rep of T relative to the basis of eigen vectors (denoted B') and taking the appropriate power of the matrix .

W6 (6.1)

Def. Orthogonal

$\v x = \[x_1\\ \vdots\\ x_n\]$ $\v y = \[y_1\\ \vdots\\ y_n\]$ be vectors.

$\v x \. \v y = x_1y_1+...+x_ny_n = \v y^T \v x = \|x\|\.\|y\|cos(\theta)$ $\theta$ is the angle between x and y.

$\v x \. \v y = \v0$ $\cos\theta = 0 \implies \theta = \f\pi2 = 90\degree$ $\v x \. \v y = \v 0$

$\v x$ $\|\v x\| = \sqrt{x_1^2 + ... + x_n^2}$ $\norm{\v x}^2 = \v x\. \v x$

$\v v$ $\v u = \f1{\norm{\v v}} \v v$

Def. Orthogonal Set

$\{\v x_1, ..., \v x_k\} \subseteq \R^n$ $\v x_i$ is not the zero vector and every pair of distinct vectors is orthogonal.

Note $\v 0$ not included so that certain properties hold (e.g. orthogonal sets are lin ind.)

Proof Orthogonal sets are linearly independent.

$c_1\v x_1 + ... + c_k\v x_k = \v 0$ $c_1 = ... = c_k = 0$

$\v x_i, 1 \le i \le k$ :

$\ba\v x_i \. (c_1\v x_1 + ... + c_k\v x_k) &= \v x_i\. \v 0 \\ c_1\ub{(\v x_i\.\v x_1)}_{=0} + c_2\ub{(\v x_i \. \v x_2)}_{=0} + ... + c_i\ub{(\v x_i \. \v x_i)}_{\ne 0} + ... + c_k\ub{(\v x_i \. \v x_k)}_{= 0} &= 0 \\ c_i &= 0 \ea$

Def. Orthonormal Set

$\norm{\v x_i} = 1 \fa x_i$

E.g. $\{\f12\[1\\1\\1\\1\], \f12\[1\\1\\-1\\-1\], \f12\[1\\-1\\1\\-1\]\}$

$\R^4$

Def. Normalize

To normalize an orthogonal set, divide each element by its length to turn it into an orthonormal set

Def. Orthogonal Complement

$W$ $\R^n$ $W$ $W^\perp = \{\v x\in\R^n:\v x\.\v w = 0 \fa \v w\in W\}$

E.g. $W$ $\R^3$ $W^\perp$ is the line through the origin perpendicular to the plane

both line and plane are vector spaces assuming they pass through the origin
every vector in the plane is orthogonal to every vector on the normal line

Facts $W$ $\R^n$

$W^\perp$ $\R^n$
$\dim W + \dim W^\perp = n$
$(W^\perp)^\perp = W$
$(\R^n)^\perp = \{\v 0\}$ $\{\v 0\}^\perp = \R^n$
$W\n W^\perp = \{\v 0\}$
$W = sp(\{\v w_1, ..., \v w_k\})$ $\v x \in W^\perp \iff \v x \. \v w_i = 0 \fa i = 1, ..., k$ (no need to check every vector in W, just check its generating set)
$\v b$ $\R^n$ $\v b = \v b_W + \v b_{W^\perp}$ $\v b_W \in W, \v b_{W^\perp} \in W^\perp$

Thm (Ortho Complement of Row/Col Space )

$A$ be an m x n matrix, then

$(row(A))^\perp = null(A)$
$(col(A))^\perp = null(A^T)$

Proof $\v x \in \R^n$

$\v x$ $(row(A))^\perp \iff \v x$ $A$ $\iff A\v x = \v 0 \iff \v x \in null(A)$
$\v x$ $(\ub{row(A^T)}_{col(A)})^\perp \iff \v x$ $A^T$ $\iff A^T\v x = \v 0 \iff \v x \in null(A^T)$

Technique to Find Ortho Complement

$A$ $W$
$null(A) = W^\perp$

E.g. $U^\perp$ $U = sp$\{\[1\\0\\0\\0\], \[0\\1\\1\\0\], \[0\\2\\0\\1\]\}$$

$\v x = \[a\\b\\c\\d\] \in U^\perp \iff \[a\\b\\c\\d\]\.\v u_i = 0$ $\bc a = 0\\ b+c = 0\\ 2b+d = 0\ec$

$u_i$ $\baug{rrrr|r}a&b&c&d\\1&0&0&0&0\\0&1&1&0&0\\0&2&0&1&0\eaug$

$\baug{cccc|c}1&0&0&0&0\\0&1&1&0&0\\0&0&1&-\frac12&0\eaug \implies \v x = \[0\\-t/2\\t/2\\t\], t \in \R$

$U^\perp = \{\v x = \[0\\-t/2\\t/2\\t\]: t \in \R\} = sp\{\[0\\-1\\1\\2\]s:s\in\R\}$

$\[a\\b\\c\\d\]\.\v u_i = 0$ $\dim(U^\perp) = \dim(V) - \dim(U)$

W8 (6.2 - 6.4)

Def. Projections (Textbook Def)

$\v d$ } be a line through the origin
$\v v$ $L$
$\v v$ into the sum of 2 vectors: 1 parallel to L and 1 perpendicular to L
$\v v = \v u + \v u^\perp = \proj_L \v v + \proj_{L^\perp} \v v$

$\v b$ $\R^n$ $\R^n$ $\v b = \ub{\v b_W} + \v b_{W^\perp}$ $\v b_W$ $\v b$ $W$

$\v b$ $W$

$\{\v v_1, \v v_2, ..., \v v_k\}$ $W$
$\{\v v_{k+1}, \v v_{k+2}, ..., \v v_{n}\}$ $W^\perp$
$\v r = \[r_1\\ \vdots\\ r_n\]$ $\v b$ $\{\v v_1, ..., \v v_n\}$ $\v b = r_1\v v_1 + ... + r_n\v v_n$
$\v b_w = r_1\v v_1 + ... + r_k\v v_k$

Proof $\R^n$ $\{w_1,...,w_k\}$ $\{w_{k+1}, ..., w_n\}$ $W^⊥$ $\{w_1,...,w_k,w_{k+1},...,w_n\}$ $\R^n$ .

$\R^n$ (which it clearly does), and is linearly independent.
$\sumto n a_iw_i = \v 0$
$v = a_1w_1 + ... + a_kw_k \implies v \in W$
$v = -(a_{k+1}w_{k+1} + ... + a_nw_n) \implies v\in W^\perp$
$v \in W, W^\perp$ $v = \v 0$ $v \. v = \v 0$ )
$a_1w_1 + ... + a_kw_k = \v 0$ $-(a_{k+1}w_{k+1} + ... + a_nw_n) = \v 0$
$\{w_{k+1}, ..., w_n\}$ $\{w_{k+1}, ..., w_n\}$ $a_1 = ... = a_n = 0$
$\implies \{w_1,...,w_k,w_{k+1},...,w_n\}$ is lin ind.

Note $\{w_1,...,w_k\}$ orthonormal $\{w_{k+1}, ..., w_n\}$ orthonormal $W^⊥$ $\{w_1,...,w_k,w_{k+1},...,w_n\}$ orthonormal $\R^n$ $\v v \in \R^n$ $\v w_i$ $\v v = \ub{(\v v\.\v w_1)\v w_1 + ... + (\v v\.\v w_k)\v w_k}_{\proj_W\v v} + \ub{(\v v\.\v w_{k+1})w_{k+1} + ... + (\v v \. \v w_n)\v w_n}_{\proj_{W^\perp}\v v}$

Proof $a_i = \v v\.\v w_i$ )

$\v v = a_1\v w_1 + ... + a_n \v w_n$ $\v v\.\v w_i = (a_1\v w_1 + ... + a_n \v w_n)\.\v w_i = a_1\ub{\v w_1\.\v w_j}_0 + ... + a_i\ub{\v w_i \. \v w_i}_1 + ... + a_n\ub{\v w_n\.\v w_i}_0 = a_i$

Def. Projections (Alternative Def)

$W$ $\R^n$ $\{\v v_1, ..., \v v_k\}$ $\v b \in \R^n$ $\proj_W \v b = \proj_{\v v_1}\v b + ... + \proj_{\v v_k}\v b$ $\d\v b_W = {\v b\.\v v_1\o \v v_1 \. \v v_1}\v v_1 + ... + {\v b\.\v v_k\o \v v_k \. \v v_k}\v v_k$

Properties $W$ $\R^n$ $\v v \in \R^n$ . Then

$\proj_W \v v \in W$
$\v v - \proj_W \v v \in W^\perp$
$\proj_W\v v$ $\v v$
Proof $\| \v v - \proj_W \v v \| \le \|\v v - \v w\|$ $\v w \in W$ $\v w = \proj_W\v v$
$\ba\norm{\v v - \v w}^2 &= \norm{\v b + \v a}^2 \\&= (\v b + \v a) \. (\v b + \v a) \\&= \v b\.\v b + 2\v a\.\v b + \v a\.\v a\\&=\norm{\v b}^2 + 0 + \norm{\v a}^2\\ &\ge \norm{\v a}^2 = \norm{\v v - \proj_W\v v}^2\ea$
$\norm{\v v - \v w} \ge \norm{\v v - \proj_W\v v} \implies$
$\norm{\v v - \v w} = \norm{\v v - \proj_W\v v}$ $\v w = \proj_W\v v$

E.g. $W = sp(\[0\\-1\\1\], \[0\\-1\\-1\])$ $W$ $\v x = \[-2\\-3\\2\]$

$\proj_W\v x$ by the projection theorem.

$\d\v x_W = {\v x\.\v w_1\o \v w_1 \. \v w_1}\v w_1 + {\v x\.\v w_2\o \v w_2 \. \v w_2}\v w_2 = \f52\[0\\-1\\1\] + \f12\[0\\-1\\-1\] = \[0\\-3\\2\]$

$\proj_W\v x$ $\v x$ $\{\[0\\1\\0\], \[0\\0\\1\]\}$

Def. Gram Schmidt Algorithm

$W$ $\R^n$ has an orthonormal basis.

$\{\v x_1, ..., \v x_k\}$ $W$ $\{\v v_1, ..., \v v_k\}$ $W$ .

$\v v_1 = \v x_1$

$\v v_2 = \v x_2 - \d{\v x_2\.\v v_1\o \v v_1\.\v v_1}\v v_1 \expl{perp. to }\v v_1$

$\v v_3 = \v x_3 - \d{\v x_3\.\v v_1\o \v v_1\.\v v_1}\v v_1 - {\v x_3\.\v v_2\o\v v_2 \. \v v_2}\v v_2\expl{perp. to }\v v_1, \v v_2$

$\vdots$

$\v v_k = \d\v x_k - {\v x_k\.\v v_1\o \v v_1\.\v v_1}\v v_1 - ... - {\v x_k\.\v v_{k-1}\o \v v_{k-1}\.\v v_{k-1}}\v v_{k-1} \expl{perp. to }\v v_1, ..., \v v_k$

$\R^2$ :

$\v v_i$ to remove fractions during the algorithm.

To get an orthonormal basis, we divide each vector by its length at the end.

Def. Orthogonal Matrix

A square matrix Q is said to be orthogonal if any of the following is true

its columns form an orthonormal set
$Q^T Q = I$
$Q\inv = Q^T$

Note $A^TA = I \ne AA^T$ . This chain of equality only holds if A is square. (They do have the same non-zero eigen values however.)

Properties

$Q\inv$ $Q^T$ are orthogonal
Proof $(Q\inv)^T = (Q^T)\inv = (Q\inv)\inv \implies Q\inv$ is orthogonal
Proof $(Q^T)\inv = (Q\inv)\inv = Q = (Q^T)^T \implies Q^T$ is orthogonal
$Q^T$ is orthogonal, rows of W also form an orthonormal set
$\pm 1$ (so orthogonal matrices are rotations or reflections)
Proof $(\det(Q))^2 = \det(Q)\det(Q) = \det(Q^T)\det(Q) = \det(Q^TQ) = \det(I) = 1$
$(Q\v x) \. (Q\v y) = \v x \. \v y$
Proof $Q\v x \. Q\v y = (Q\v y)^T(Q\v x) = \v y^T(Q^TQ)\v x = \v y^T \v x = \v x \. \v y$
$\norm{Q\v x} = \norm{\v x}$
Proof $\|Q\v x\|^2 = (Q\v x)\.(Q\v x) = \v x \. \v x = \norm{\v x}^2$
$\v x$ $\v y$ $(Q\v x)$ $(Q\v y)$
Proof $\cos\theta = \d{Q\v x\.Q\v y\o \norm{Q\v x}\norm{Q\v y}} = {\v x \. \v y\o\norm{\v x}\norm{\v y}}$
$\lambda$ $|\lambda| = 1$ (all eigs are on the unit circle in the complex plane)
Proof $Q\v x = \lambda \v x$ $\norm{Q\v x} = \norm{\lambda \v x}$ $\norm{Q\v x} = \norm{\v x}$ $\norm{\lambda\v x} = \norm{\v x} \implies \norm{\lambda} = 1$
$Q$ $R$ $QR$
Proof $(QR)^T(QR) = R^TQ^TQR = R^TR = I$

Thm (Relation to Change of Basis)

$B = \{\v w_1, ..., \v w_n\}$ $\R^n$ ,

$E$ $[I]_B^E = \[\v w_1\mid...\mid\v w_n\]$ , which is an orthogonal matrix.

$([I]_B^E)\inv = ([I]_B^E)^T \expl{($Q\inv = Q^T$)}$

$\norm{\v v_B} = \norm{\ub{[I]_B^E}_Q\ \ub{\v v_E}_\v v} = \norm{\v v_E} \expl{(use Qv = v, proves Q preserves length)}$

E.g. $(\v a_1, \v a_2, \v a_3, \v a_4)$ $\R^4$ $[2, 1, 4, -3]$ $\v b$ $\R^4$ $\norm {\v b}$ .

$\epsilon$ $\R^4$ $B = (\v a_1, \v a_2, \v a_3, \v a_4)$ $[2, 1, 4, 3] = [\b b]_B$
$\|\mathrm{b}\|=\left\|[\mathrm{b}]_{\varepsilon}\right\|=\left\|[I]_{B}^{\varepsilon}[\mathrm{b}]_{B}\right\|=\left\|[\mathrm{b}]_{B}\right\|=\|[2,1,4,-3]\|=\sqrt{30}$

Def. Orthogonal Transformation

$T:\R^n\to\R^n$ $T(\v v) \. T(\v w) = \v v\. \v w \fa \v v, \v w \in \R^n$

$\iff$ its matrix is orthogonal)

Def. Projection Matrix

$W$ $\R^n$ $T:\R^n\to \R^n$ $T(\v x) = \proj_W \v x$ $T$ projection matrix $W$ .

Steps $W$ )
$W$ $A = \[\v w_1 \mid ...\mid\v w_k\]_{n\times k}$ $W$ $A$
$P = A(A^TA)\inv A^T$ $W$ $\proj_W\v x = P\v x$
$\{\v w_1, ..., \v w_k\}$ $P = AA^T$

E.g. $W = sp(\[\frac1{\sqrt3}\\ -\frac1{\sqrt3}\\ \frac1{\sqrt3}\], \[\frac1{\sqrt2}\\ \frac1{\sqrt2}\\ 0\])$ $\proj_W\v x$

$A = \[\frac1{\sqrt3} &\frac1{\sqrt2}\\ -\frac1{\sqrt3} &\frac1{\sqrt2}\\ \frac1{\sqrt3} &0\]$ $A^TA = I$

$P = A(\ub{A^TA }_{I})\inv A^T = AA^T = \[\frac56&\frac16&\frac13\\ \frac16&\frac56&-\frac13\\\frac13&-\frac13&\frac13\]$ $\proj_W\v x = [P]\[x_1\\x_2\\x_3\] = \f16\[5x_1 + x_2 + 2x_3\\x_1 + 5x_2 - 2x_3\\2x_1 - 2x_2 + 2x_3\]$

Proof $P = A(A^TA)\inv A^T$ )

$A\v x = \[w_1 \mid ... \mid w_k\]\[x_1\\ \vdots\\ x_k\] = \sum_i x_i\v w_i$

$\implies \proj_W\v b \in W \implies \proj_W\v b \in col(A)$

$\implies \ex \hat x \in \R^k$ $\proj_W\v b = A\hat x$

$\implies b - \proj_W\v b = \v b - A\hat x$ $A\v x$

$\implies (b - A\hat x)\.(A\v x) = 0$

$\implies (A\v x)^T(\v b - A\hat x) = 0 \expl{since } x\.y = y^Tx$

$\implies (\v x^T)(A^T\v b - A^TA\hat x) = 0 \expl{since }(A\v x)^T = \v x^TA^T$

$\implies (A^T\v b - A^TA\hat x)\. \v x = 0 \expl{write as dot product}$

$\implies A^T\v b - A^TA\hat x = \v 0 \expl{can now solve for }\hat x$

$\implies \hat x = (A^TA)\inv A^T\v b \expl{since $rank(A^TA) = rank(A) = k$, $(A^TA)_{k\times k}$ has full rank so must be invertible}$

$\implies \proj_W \v b = A\hat x = \ub{A(A^TA)\inv A^T}_P \v b$

Properties

Every n x n matrix that is both symmetric and idempotent is a projection matrix.

$P^T = P$
$P^2 = P$

E.g. Find all invertible projection matrices.

$P^{2}=P .$

$P$ $P^{2}=P$ $P^{-1}$ $P=I$ .

W9 (9.1, 9.2)

Def. Least Squares

$A\v x = \v b$ $\v b$ is not in col(A).

$A\v x = \v b$ $\hat x$ $\|b - A\hat x\| \le \|b - A\v x\| \fa \v x \in \R^n$

$\hat x$ $e = \|A\hat x - \v b\|^2$ $m \ge n$ )

$A\v x = \v b$ $A^T$ $A^TA\hat x = A^T \v b$ $\hat x = (A^TA)\inv A^T\v b$ .

Justification $\hat x$ $A\hat x \in W = col(A)$ .

$(A\hat x - \v b) \perp col(A) \implies (A\hat x - \v b) \in (col(A))^\perp = null(A^T) = \{\v x \in \R^n:A^T\v x = 0\}$

$\implies A^T(A\hat x - \v b) = \v 0 \implies A^TA\hat x = A^T \v b$

E.g. Find the line closest to points (0, 6) (1, 0) and (2, 0)

$\implies \bc 0m+b=6\\1m+b=0\\2m+b=0 \ec$ $A\v x = \v b \implies \[0&1\\1&1\\2&1\]\[m\\b\] = \[6\\0\\0\]$

$A^TA\hat x = A^T \v b \implies \hat x = (A^TA)\inv A^T\v b = $\[0&1&2\\1&1&1\]\[0&1\\1&1\\2&1\]$\inv \[0&1&2\\1&1&1\]\[6\\0\\0\] = \[-3\\5\]$

So y = -3x + 5 is the line of best fit

Note $A^TA$ is invertible by the following thm.

Thm (Invertibility, Lin Ind, and Uniqueness)

$\v b \in \R^m$ . Then the following are equivalent:

$A\v x = \v b$ $\hat x$
columns of A are linearly independent
$A^TA$ is invertible

Def. Symmetric Matrices

$A = A^T$ always has real eigen values and is always diagonalizable (by a real orthogonal matrix).

Properties

Eigenspaces of a symmetric matrix are mutually orthogonal.
Thus, symmetric matrices have n mutually perpendicular eigen vectors.

Proof $\lambda \in \C$ $A$ be a real matrix.

$A\v x = \lambda\v x$ $\v x \ne \v 0$

$A$ $A = \overline A = \ol{A^T}$

$\overline{\v x^T} A\v x = \overline{\v x^T} \lambda\v x = \lambda$\overline{\v x ^T}\v x$$

$\overline {\v x^TA^T}\v x = \overline{(A\v x)^T}\v x = \overline{(\lambda\v x)^T}\v x = \overline \lambda$\overline{\v x ^T}\v x$$

$\lambda = \overline \lambda \implies \lambda$ is real

Def. Orthogonally Diagonalizable

orthogonally diagonalizable $Q$ $D$ $A = QDQ^T \iff Q^TAQ = D$

Note $\iff$ orthogonally diagonalizable

E.g. $1&1\\0&2$ is diagonalizable but not orthogonally diagonalizable since it is not symmetric.

$A$ :
Find the eigen values and eigen vectors
Form a basis with the eigen vectors and normalize
Form Q with the normalized eigen vectors as columns

E.g. $A = \[1&2\\2&-2\]$ using an orthogonal matrix.

$\lambda_1 = -3$ $\v v_1 = \[1\\-2\]$ $\lambda_2 = 2$ $\v v_2 = \[2\\1\]$

$A = PDP\inv$ $P = \[\v v_1 | \v v_2\] = \[1&2\\-2&1\]$ $D = \[-3&0\\0&2\] \expl{regular diagonalization}$

$\{\v v_1, \v v_2\}$ $\{\[\frac1{\sqrt5}\\ -\frac2{\sqrt5}\] \[\frac2{\sqrt5}\\ \frac1{\sqrt5} \]\}$ , then we can use

$A = QDQ\inv$ $Q = \[\frac1{\sqrt5} &\frac2{\sqrt5}\\ -\frac2{\sqrt5} &\frac1{\sqrt5} \]$ $\expl{orthogonal diagonalization}$

$Q$ $Q\inv = Q^T$ $A = QDQ^T$ .

Def. Complex Numbers

cartesian $z = a + bi$

polar $z = r(\cos\theta + i\sin\theta) = re^{i\theta}$

modulus $|z| = \sqrt{a^2+b^2} = r$

principal argument $\arg(z)$ $\theta$ $-\pi < \theta \le \pi$

complex conjugate $z = a+ib$ $\bar z = a-ib$ $z\ol z = |z|^2$ $a^2 + b^2 > 0$

Thm $z$ $z = \overline z$

Proof $z=a+b i$ $a, b \in \mathbb{R}$ $\bar{z}=a-b i$ .

$(\Rightarrow)$ $z$ $b=0 \implies z=a=\bar{z}$ .

$\Leftarrow$ $z = \overline z$ $a + bi = a - bi \implies b = -b \implies b = 0$

E.g. $z=\sqrt{3}-i$

$r = |z|=\sqrt{(\sqrt{3})^{2}+(-1)^{2}}=\sqrt{4}=2$

$\arg(z) = \theta=-\pi / 6$

E.g. $z\inv$ in the form a + bi where a and b are real numbers.

$\d\frac{1}{-1+i}=\frac{1}{-1+i} \frac{-1-i}{-1-i}=\frac{-1-i}{1+1}=-\frac{1}{2}-\frac{1}{2} i$

Def. De Moivre's Theorem

$(e^{i\theta})^n = e^{in\theta}$ $(re^{i\theta})^n = r^ne^{in\theta}$ where n is a positive integer

Def. nth roots of z

E.g. $z^4 = 2(\sqrt3 i - 1)$

$-2 + 2\sqrt 3i \implies r = \sqrt{4 + 4\.3} = 4, b_\triangle = 2, h_\triangle = 2\sqrt 3 \implies \theta = \f{2\pi}3$

$4e^{i({2\pi\o3})}$

$(re^{i\theta})^4$

$(re^{i\theta})^4 = 4e^{i({2\pi\o3})} \implies r^4e^{i\theta4} = 4e^{i({2\pi\o3})}$

$r^4 = 4 \implies r = \sqrt 2$ $4\theta = \f{2\pi}3 + 2k\pi \implies \theta = \f\pi6 + \f\pi2k, k\in \Z$

$\implies z = \sqrt 2 e^{(\pi/6)i} = \sqrt2(\f{\sqrt3}2 + \f12i)$

$\implies z = \sqrt 2 e^{(2\pi/3)i} = \sqrt2(- \f12 + \f{\sqrt3}2i)$

$\implies z = \sqrt 2 e^{(7\pi/6)i} = \sqrt2(-\f{\sqrt3}2 - \f12i)$

$\implies z = \sqrt 2 e^{(5\pi/3)i} = \sqrt2(\f12 - \f{\sqrt3}2i)$

Formula $z = r(\cos \theta + i\sin\theta)$ :

$r^\frac1n $\cos\(\f\theta n + \f{2k\pi}n$ + i\sin$\f\theta n + \f{2k\pi}n$\)$ for k = 0, 1, 2, …, n-1

E.g. Find the 4 fourth roots of 1.

Method 1: Factor and solve.
$z^{4}-1=0 \quad \rightarrow \quad\left(z^{2}-1\right)\left(z^{2}+1\right)=0 \quad \rightarrow \quad z=\pm 1$ $z=\pm i$ .
$\theta = 0, r = 1, n = 4$
$\ba k&=0 \rightarrow \cos 0+i \sin 0=1 \\ k&=1 \rightarrow \cos \frac{\pi}{2}+i \sin \frac{\pi}{2}=i \\ k&=2 \rightarrow \cos \pi+i \sin \pi=-1 \\ k&=3 \rightarrow \cos \frac{3 \pi}{2}+i \sin \frac{3 \pi}{2}=-i \ea$

E.g. $A = \[1&-1\\1&1\]$

$p(\lambda) = (\lambda - 1)^2 + 1 = \lambda^2 - 2\lambda + 2 = 0$
$\lambda = {2\pm\sqrt{4 - 8}\o2} = 1\pm i$
$\lambda_1 = 1 + i$
$A - \lambda I = \[1 - (1 + i) &-1\\1 &1-(1+i)\] = \[-i &-1\\1 &-i\]$
$\bc R_1 = iR_1 = [1, -i]\\R_2 = R_2 - i(R_1) = [1+i^2 = 0, -i+i = 0]\ec \implies \[1&-i\\0&0\] \implies a = ib \implies v_1 = \[i\\1\]$
$\lambda_2 = 1-i$
$v_2 = \overline{v_1} = \[-i\\1\]$

Def. Complex Vector Space

$\C^n =$ $\C^n$ $a_1+b_1i\\ \vdots\\ a_n+b_ni$ $\C$

Real scalars	Complex scalars
$\dim(\C^n) = 2n$	$\dim(\C^n) = n$
$\{\[1\\0\\\vdots\\0\],..., \[0\\\vdots\\0\\1\], \[i\\0\\\vdots\\0\],..., \[0\\\vdots\\0\\i\]\}$	$\{\[1\\0\\\vdots\\0\],..., \[0\\\vdots\\0\\1\]\}$

A complex vector space is one in which scalars are complex numbers.

E.g. $M_{2, 2}(\C) = \{\[a&b\\c&d\]:a, b, c, d \in \C\}$ is a complex vector space (with usual operations)

E.g. $\R^2$ $v\in \R^2, i\.v\notin \R^2$ )

E.g. $\v v = \[-1\\1\]$ $\v v_B$ $B = $\[i\\i\],\[i\\1\]$$ $\C^2$

$-1\\1\] = z_1\[i\\i\] + z_2\[i\\1$
$\baug{rr|r} i&i&-1\\i&1&1\eaug \to \baug{rr|r}i&i&-1\\0&1-i&2\eaug \to \baug{rr|r} 1&1&i\\0&1-i&2\eaug \to \baug{rr|r} 1&1&i\\0&1&{2\o 1-i}{1+i\o 1+i} =1+i \eaug \to \baug{rr|r}1&0&-1\\0&1&1+i\eaug \implies$
$\bc z_1 = -1\\ z_2 = 1+i \ec\implies v_b = \[z_1\\z_2\] = \[-1\\1+i\]$
Note $\{\[i\\i\], \[i\\1\]\}$ is linearly independent since the matrix has rank 2.

Def. Conjugate Transpose

$A^* = \ol A^T$

Properties

$k \in \C$

$(A^*)^* = A$
$(A+B)^* = A^* + B^*$
$(kA)^* = \ol kA^*$
$(AB)^* = B^*A^*$ $A, B$ are square

Def. Euclidean Inner Product / Dot Product

$\R^n: \v x \. \v y = x_1y_1 + ... + x_ny_n = \v y^T\v x$ $\|\v x\|^2 = \v x\. \v x$

$\C^n:\langle\v x, \v y\rangle = x_1\ol{y_1} + ... + x_n\ol{y_n} = \v y^*\v x$

E.g. $\langle u, v\rangle$ $u = \[i\\0\\1+i\]$ $v = \[3-i\\7\\2i\]$

$\langle u, v \rangle = (i)(\ol{3-i}) + (1+i)(\ol{2i}) = (i)(3+i) + (1+i)(-2i) = 3i -1 - 2i +2 = 1 + i$

Complex Dot Product Properties

$u, v, w \in \C^n, k \in \C$

$\langle u, u \rangle \ge 0$ $\<u, u\> = 0$ $u = \v 0$
$\<u, v\> = \ol{\<v, u\>} \expl{not commutative! order matters!}$
$\<u+v, w\> = \<u, w\> + \<v, w\>$ $\<w, u+v\> = \<w, u\> + \<w, v\>$
$\<ku, v\> = k\<u, v\>$ $\<u, kv\> = \ol k \<u, v\>$

Def. Euclidean Norm

$u \in \C^n$ $||u||= \sqrt{\<u, v\>}$

Def. Unitary Matrix

$U$ analogous to orthogonal matrices $\R^n$ )

$U$ $\C^n$
$U\inv = U^*$
$UU^* = I$

Property $AA^* = BB^* = I \implies AB(AB)^* = ABB^*A^* = AIA^* = I$

Def. Hermitian Matrix

$H$ $H = H^*$ $\R^n$ ).
It has real eigen values and is diagonalizable (by a unitary matrix).
Eigen vectors corresponding to distinct eigen values are orthogonal.
$A = A^* \iff \<Az, w\> = \<z, Aw\> \fa w, z, \in \C^n$

Proof (eigen values are real)

$\lambda \in \C$ $H$ be a Hermitian matrix.

$H\v x = \lambda\v x$ $\v x \ne \v 0$

$H$ $H = H^*$

$\overline{\v x^T} H\v x = \overline{\v x^T} \lambda\v x = \lambda$\overline{\v x ^T}\v x$$

$\overline {\v x^T}H^*\v x = \overline{(H\v x)^T}\v x = \overline{(\lambda\v x)^T}\v x = \overline \lambda$\overline{\v x ^T}\v x$$

$\lambda = \overline \lambda \implies \lambda$ is real

Proof (eigen vectors are orthogonal)

$\lambda_1, \lambda_2$ $A$ $v_1$ $v_2$ $Av_1= \lambda_1 v_1, Av_2 = \lambda v_2$

$\<Av_1, v_2\> = \<\lambda_1 v_1, v_2\> = \lambda_1\<v_1, v_2\>$ $\<Av_1, v_2\> = \<v_1, Av_2\> = \<v_1, \lambda_2 v_2\> = \ol\lambda_2\<v_1, v_2\>$ $\lambda_1 = \ol\lambda_2$

$v_1 = v_2$ $\lambda_1 = \lambda_2$ $\lambda_1 = \ol\lambda_1, \lambda_2 = \ol\lambda_2$ which also proves the eigs are real)

$\ol\lambda_2 = \lambda_2 \implies \lambda_1\<v_1, v_2\> = \lambda_2\<v_1, v_2\> \implies (\lambda_1 - \lambda_2)\<v_1, v_2\> = 0$

$\lambda_1 \ne \lambda_2$ $\<v_1, v_2\> = 0 \implies v_1 \perp v_2$

Proof $\<Az, w\> = \<z, Aw\>$ )

$\Rightarrow$ $A = A^*$ $\<Az, w\> = w^*(Az) = W^*A^*z = (Aw)^*z = \<z, Aw\>$

$\Leftarrow$ $A = [a_{ij}]$ $a_{ij} = e_i^TAe_j = e^*_iAe_j = \<Ae_j, e_i\> = \<e_j, Ae_i\> = (Ae_i)^*e_j = e_i^TA^*e_j = \ol{a_{ji}}$

Note $A$ $A^* = -A$ $iA$ is Hermitian, and vice versa.

Fact Every square matrix can be decomposed into the sum of a Hermitian matrix and skew-Hermitian matrix

$C = \f12\ub{(C + C^*)}_{Hermitian} + \f12\ub{(C - C^*)}_{skew-Hermitian}$

Def. Normal Matrix

$AA^* = A^*A$

Property

$A$ $\|A\b z\| = \|A^*\b z\| \fa \b z \in \C^n$
Proof $\|A\b z\|^2 = \ = (Az)^*(Az)= z^*A^*Az = z^*AA^*z = (A^*z)^*(A^*z) = \ = \|A^*z\|^2$

Shortcuts (to prove a matrix is normal)

Every Hermitian matrix is normal.
Proof $H=H^*$ $HH^* = H^2$ $H^*H = H^2 \implies HH^* =H^2 = H^*H$
Every unitary matrix is normal.
Proof $UU^* = I \iff U^* = U\inv$ $UU^* = UU\inv = I = U\inv U = U^*U$
Note for square matrices A and B, AB = I = BA always holds
Every skew-Hermitian matrix is normal.
Proof $A^*A =-AA = -A^2$ $AA^* = A(-A) = -A^2$

Def. Unitarily Diagonalizable

$A$ $A$ is a normal matrix.

E.g. $a \in \C$ $A = \[i&a\\2&i\]$ is unitarily diagonalizable.

$AA^* = A^*A$
$i&a\\2&i\]\[-i&2\\\ol a&-i\] = \[-i&2\\\ol a&-i\]\[i&a\\2&i$
$\implies i(-i) + a\ol a = i(-i)+ 4 \implies 4 = a\ol a \implies 4 = |a|^2 \implies |a| = 2$
$a$ $x + iy$ $x^2 + y^2 = 4$

E.g. $a \in \C$ $i&4\\a&i$ is unitarily diagonalizable.

$AA^* = A^*A$
$i&4\\a&i\]\[-i&\ol a\\4&-i\] = \[-i&\ol a\\4&-i\]\[i&4\\a&i$
$-1 + 16 = -1 + a^2 \implies |a| = 4$
$a$ $x + iy$ $x^2 + y^2 = 4$

W10 (9.3)

Comparison - Real vs Complex

Real (Symmetric matrix)	Complex (Hermitian matrix)
eigen values of A are real	eigen values of H are real
$\R^n$	$\C^n$
orthogonal $Q$ $Q^TAQ$ is diagonal	unitary $U$ $U^*HU$ is diagonal

Notes (eigenvalues)

If a matrix has odd order (order = m x n), then the eigen values must be real.

conjugate pairs $a+ib$ $a-ib$ is also an eigen value.

For an orthogonal matrix, the eigen value pairs all lie on the unit circle.

For a Hermitian matrix, eigen values lie on the real axis; for a skew-Hermitian matrix, eigen values lie on the imaginary axis.

Proof $ib, b\in \R$ )

$\v v$ $A$ $\lambda = a + ib$

$\<Av, v\> = v^*Av = v^*\lambda v = \lambda v^*v$

$\<Av, v\> = v^*(-A^*)v = -(Av)^* v = -(\lambda v)^* v = -\ol\lambda v^*v$

$\lambda = -\ol \lambda \implies a+ib = -(\ol{a + ib}) = -(a-ib) = -a + ib \implies a = -a \implies a = 0$

$\<Av, v\>$ $\<Av, w\>$ $\<v, w\> = 0$ (eigen vectors corresp to dif eigen values are orthogonal)

Def. Schur's Theorem

$A$ is an n x n complexunitary $U$ $U\inv AU = U^*AU = T$ $T$ $A$ )

Def. Spectral Theorem

Recall Let A be an n x n real matrix. A is orthogonally diagonalizable iff A is symmetric.

Let A be an n x n complex matrix. A is unitarily diagonalizable iff A is normal.

Lemma Hermitian $U$ $U^*AU$ is diagonal.

Proof $T = U^*AU$ .

$T^* = (U^*AU)^* = U^*A^*(U^*)^* = U^*AU = T \implies T^* = T$

Since T is both upper triangular and lower triangular, it must be diagonal

Note The converse is false.

E.g. $A = \[0&1\\-1&0\]$ is not Hermitian but it is unitarily diagonalizable.

W11 (9.4)

Def. Jordan Canonical Form

$A$ $\lambda_i$ 's are not necessarily distinct).

The matrix is composed of Jordan blocks along the diagonal.

Def. Block Matrix

A block matrix is one that has been partitioned into blocks.

We can multiply block matrices (assuming sizes compatible):
$A&B\\C&D\]\[E&F\\G&H\] = \[AE+BG&AF+BH\\CE+DG&CF+DH$ Note that order matters!
$\det$\[A&B\\0&C\]$ = \det A \. \det C$
A block diagonal matrix has square blocks down its diagonal. (JCF is block diagonal)
$\det A = \det(A_1)\det(A_2)...\det (A_n)$

E.g. $A = \[0&1&*&*\\-1&0&*&*\\0&0&2&3\\0&0&0&4\]$

$A$ can be partitioned into 2 x 2 blocks.

$\pm i$ , and the bottom right block has 2 and 4.

Properties of Jordan Blocks

$J$ $\lambda &1 & 0 & ... & 0\\0&\lambda & 1 &\ddots &\vdots\\\vdots&\ddots &\lambda&\ddots&0\\\vdots&&\ddots&\ddots&1\\0&...&...&0&\lambda$

$\lambda$ with algebraic mult k
Proof $p(x) = \det(J - xI) = (\lambda - x)^k \implies x = \lambda$ is a root with mult K
$\dim(E_\lambda) = 1$ (if k not = 1, the matrix is not diagonal)
Proof $J - \lambda I = \[0 &1 & 0 & ... & 0\\0&0 & 1 &\ddots &\vdots\\\vdots&\ddots &0&\ddots&0\\\vdots&&\ddots&\ddots&1\\0&...&...&0&0\]$ $E_\lambda = null(J-\lambda I) = sp$\[1\\0\\\vdots\\0\]$ = sp(e_1)$
$(J - \lambda I)^P = O$ $P = k$
$(J - \lambda I)^P = O$ $i < k$ $(J - \lambda I)^i \ne O$
Proof $J - \lambda I$
E.g. $A = \[0&1&0&0\\0&0&1&0\\0&0&0&1\\0&0&0&0\]$ so k = 4
$A^2 = \[0&0&1&0\\0&0&0&1\\0&0&0&0\\0&0&0&0\]$ $A^3 = \[0&0&0&1\\0&0&0&0\\0&0&0&0\\0&0&0&0\]$ $A^4 = \[0&0&0&0\\0&0&0&0\\0&0&0&0\\0&0&0&0\]$

Fact

$(J-\lambda I)e_i = e_{i-1}$ $1 < i \le k$

$(J - \lambda I)e_1 = 0 \implies e_1 \in null(J - \lambda I)$ $e_1$ is an eigen vector with rank 1

$(J - \lambda I)e_2 = e_1 \implies (J - \lambda I)^2 e_2 = (J - \lambda I)e_1 = \v 0 \implies e_1, e_2 \in null(J-\lambda I)^2$

$\vdots$

$(J - \lambda I)^p e_p = \v 0 \implies e_1, e_2, ..., e_p \in null(J - \lambda)^p$

Def. Generalized Eigenvector

$A$ $\v x \in \C^n$ $\lambda$ $(A - \lambda I)^p\v x = \v 0$ for some positive integer p.

$\lambda$ $\v 0$ $\hat E_\lambda = \{\v x \in \C^n : (A - \lambda I)^p\v x = \v 0\t{ for some }p\}$

Notes

ordinary eigenvectors are generalized eigenvectors with rank 1.
$A$ $E_{\lambda_1} \oplus ... \oplus E_{\lambda_r} \subset \C^n$
$\hat E_{\lambda_1} \oplus ... \oplus \hat E_{\lambda_r} = \C^n$
$\dim(\hat E_\lambda) =$ $\lambda$

Why find generalized eigenvectors?

$A$ $P\inv AP$

E.g. $A = \[1&1&0\\0&1&2\\0&0&3\]$

Eigen values are 1, 1, 3

$\lambda = 3 \implies E_3 = sp$\[1\\2\\2\]$$

$\lambda = 1 \implies E_1 = null$\[0&1&0\\0&1&2\\0&0&3\]$ = sp(e_1)$

$p = 1$ $(A - \lambda I)\v x = \v 0 \implies \v x = e_1$

$p = 2$ $(A - \lambda I)^2\v x = \v 0 \implies \v x = null((A - \lambda I)^2) = null$\[0&1&0\\0&0&2\\0&0&3\]^2$ = null$\[0&0&1\\0&0&0\\0&0&0\]$$

$= \{\[s\\t\\0\]: s, t \in \C\} = sp(e_1, e_2)$ $e_2$ is a generalized eigen vector

Thm (Similar to JCF)

Every square complex matrix is similar to a JCF.

Properties

JCF is unique (up to permutation of blocks)
$\lambda$ $A$ generates a block)
$\lambda$ is independent of other eigenvalues

E.g. $A$ has eigenvalues {1, 1, 1, 2, 2, 2, 2}, what are the possible JCF's?

$\lambda = 1$ , the possible configurations are:

$\lambda = 2$ , the possible configurations are:

For small order, the geo mult gives the block structure, but it does not tell us the structure for alg mult = 4, geo mult = 2 (see above underbrace). How do we tell which structure to use?

$\dim(null(A-\lambda I)^k)$ $k \ge 1$ , i.e. generalized eigenspaces.

Note Each k x k Jordan block has

$\lambda$ $\dim(E_\lambda) = \dim(null(J - \lambda I)) = 1$
$e_1, e_2, ..., e_k$
Note $(A - \lambda I)^P \v x = \v 0$ $(A - \lambda I)^{p+1}\v x = \v 0$

In general, an arbitrary matrix has a chain for each block in its JCF.

E.g. For a 5x5 matrix, an eigen value with alg mult = 5, geo mult = 3, we need 2 generalized eigenvectors:

3 chains – either 1 of them has 2 generalized eigenvectors, or 2 of them have 1 generalized eigenvector

E.g.

$\dim(null(B)) = 9$ for the last one

E.g.

$A = \left(\begin{array}{@{}c|c@{}} \begin{matrix} \lambda & 1 & 0 \\ 0 & \lambda & 1 \\ 0 & 0 & \lambda \end{matrix} & \bigzero \\ \hline \bigzero & \begin{matrix} \lambda & 1 \\ 0 &\lambda \end{matrix} \end{array}\right)$

$v_1 = \[1\\0\\0\\ \hline0\\0\] \to v_2 = \[0\\1\\0\\\hline0\\0\] \to v_3 = \[0\\0\\1\\\hline0\\0\]$
$w_1 = \[0\\0\\0\\\hline1\\0\]\to w_2 = \[0\\0\\0\\\hline0\\1\]$
$v_1, w_1 \in null(A - \lambda I)$ are independent eigenvectors or rank 1
$v_2, w_2 \in null((A - \lambda I)^2), \notin null(A - \lambda I)$ are independent generalized eigenvectors of rank 2
$v_3 \in null((A - \lambda I)^3), \notin null((A - \lambda I)^2)$ is an independent generalized eigenvector of rank 3
$P = \[v_1 | v_2 | v_3 | w_1 | w_2\]$ $J = P\inv A P$
Fact $\dim (null(A - \lambda I)^k) =$ # ind. gen. eigenvectors of rank at most k
$\dim (null(A - \lambda I)) = 2$ $v_1, w_1$
$\dim (null(A - \lambda I)^2) = 4$ $v_1, v_2, w_1, w_2$
$\dim (null(A - \lambda I)^3) = 5$ $v_1, v_2, v_3, w_1, w_2$
$\dim (null(A - \lambda I)^k) = 5$ $k \ge 3$

Fact $\dim(null(A-\lambda I)^k) - \dim(null(A - \lambda I)^{k-1})$ $\ge k$
# $\ge k$ $\ge k+1$ )
$=\dim(null((A - \lambda k)^k)) - \dim(null((A - \lambda k)^{k-1})) - \big(\dim(null((A - \lambda k)^{k+1})) - \dim(null((A - \lambda k)^{k}))\big)$
$= 2\dim(null(A - \lambda I)^k) - \dim(null(A - \lambda I)^{k-1}) - \dim(null(A - \lambda I)^{k+1})$

E.g. $A$ $p(\lambda) = (\lambda - 3)^{13}$ and suppose:

$\dim (null((A - 3I)^1)) = 6$ so # ind. gen. eigenvectors of rank at most 1 = 6

$\dim (null((A - 3I)^2)) = 10$

$\dim (null((A - 3I)^3)) = 12$

$\dim (null((A - 3I)^4)) = 13$

$A$ $\lambda = 3$ with alg mult 13
Obtain 6 chains, each corresponding to a Jordan block (each eigen vector produces a Jordan block), and the length of the chain = size of the block
$2\dim(null(A - \lambda I)^k) - \dim(null(A - \lambda I)^{k-1}) - \dim(null(A - \lambda I)^{k+1})$
$\dim (null((A - 3I)^1)) = 6$ 2(6) - 0 - 10 = 2 blocks of size 1
$\dim (null((A - 3I)^2)) = 10$ 2(10) - 6 - 12 = 2 blocks of size 2
$\dim (null((A - 3I)^3)) = 12$ 2(12) - 10 - 13 = 1 block of size 3
$\dim (null((A - 3I)^4)) = 13$ 2(13) - 12 - 13 = 1 block of size 4
$\dim (null((A - 3I)^5)) = 13$ 2(13) - 13 - 13 = 0 block of size 5

E.g. $p(\lambda) = (\lambda -7)^5$ and

$\dim (null((A - \lambda I)^2)) = 2$

$\dim (null((A - \lambda I)^3)) = 3$

$\dim (null((A - \lambda I)^4)) = 5$

Def. Cayley Hamilton Thm

Every square matrix satisfies its own characteristic polynomial.

E.g. $A = \[1&2\\3&4\]$ $p(\lambda) = \lambda^2 - 5\lambda - 2= 0$ $A\inv$ (in terms of powers of A).

$p(A) = A^2 - 5A - 2I = \[0&0\\0&0\]$

$\implies 2I = A^2 - 5A \implies 2A\inv = A - 5I \implies A\inv = \f12 (A - 5I) = \f12\[-4&2\\3&-1\]$

E.g. $A = \[2&1\\5&-2\]$ $A^6$ .

$p(\lambda) = \lambda^2 - 9 \implies p(A) = A^2 - 9I = 0 \implies A^2 = 9I \implies (A^2)^3 = (9I)^3 \implies A^6 = 729I = \[729 &0\\0&729\]$

MATB24

W1 (3.1)

Def. Vector Space

Thm 3.1 (Elementary Properties of Vector Spaces)

W2 (3.2)

Def. Linear Combination

Def. Span

Thm (Span of a subset is a subspace)

Def. Subspace

Thm 3.2 (The Subspace Test)

Def. Linear (In)dependence

Def. Basis

Def. Dimension

Thm (Dimension and Linear Independence)

Thm (Lin Ind Set ⊆\subseteq Basis ⊆\subseteq Spanning Set)

Thm 3.3 (Unique Representation)

Def. Finitely Generated

Thm (Shortcut for proving basis)

W3 (3.3, 3.4 pg. 213-216)

Def. Coordinate Vector

Thm (Coordinatization of Vector Spaces)

Thm (Dimension Determines Structure)

Def. Linear Transformation

Def. Domain, Codomain

Def. Image, Inverse Image

Def. Range, Kernel / Null Space

Thm (Rank-Nullity)

Technique to Find Bases for Image(T)

Def. Row Space

Def. Column Space

Strategy for finding basis of row(A) and col(A)

Def. Rank

W4 (remainder of 3.4)

Thm 3.7 (Lin transf preserves subspaces)

Def. Onto & 1-1

Thm (Compositions of lin transf is a lin transf)

Def. Invertible Transformation

Thm (T is invertible if A is invertible)

Thm 3.8 (T is invertible iff 1-1 and onto)

Def. Isomorphism

Thm (Iso proof shortcut if same dim)

Thm 3.9 (Coordinatization is an isomorphism)

W5 (7.1, 7.2)

Def. Standard Matrix Rep of Lin Transf

Matrix Rep Notation

Def. Matrix of T Rel. to Basis B

Def. Matrix of T Rel. to Bases B and B'

Def. Change of basis matrix

Def. Similar Matrices

Thm 7.1 (Basis Change in A Vector Space)

Shortcut to find Matrix Rep (Add basis E)

Thm (Basis Change between 2 Vector Spaces)

Def. Eigen Values/Vectors

Def. Eigen Space

Def. Alg & Geo Multiplicities

Def. Diagonalizable

W6 (6.1)

Def. Orthogonal

Def. Orthogonal Set

Def. Orthonormal Set

Def. Normalize

Def. Orthogonal Complement

Thm (Ortho Complement of Row/Col Space )

Technique to Find Ortho Complement

W8 (6.2 - 6.4)

Def. Projections (Textbook Def)

Def. Projections (Alternative Def)

Def. Gram Schmidt Algorithm

Def. Orthogonal Matrix

Thm (Relation to Change of Basis)

Def. Orthogonal Transformation

Def. Projection Matrix

W9 (9.1, 9.2)

Def. Least Squares

Thm (Invertibility, Lin Ind, and Uniqueness)

Def. Symmetric Matrices

Def. Orthogonally Diagonalizable

Def. Complex Numbers

Def. De Moivre's Theorem

Def. nth roots of z

$\subseteq$ $\subseteq$ Spanning Set)